Optimizing the Crawl Budget Depending on the number of products and category pages, a Magento store can quickly become very large. It’s not uncommon for many Magento-based online stores to eventually have more than 10,000 indexable pages on their website. When combined with a series of filters, this usually results in crawling issues. That’s why it’s important to efficiently guide search engine crawlers through your online store and manage your crawl budget effectively. Â
Missing canonical URLs It’s easy to add canonical URLs to categories and products through Magento’s settings. However, this does not add canonical URLs to other pages, such as static pages and blogs. Developers must therefore configure this themselves.
The charging speed Many Magento stores struggle to achieve good loading speeds. As a result, they fail to meet the Core Web Vitals standards. But more importantly, your visitors end up on a store that loads too slowly. Many factors influence loading speed, such as your CDN, theme, and extensions. With the right setup and a good theme (custom-built or, for example, Hyvä), you can ensure a smooth loading experience.
Large amount of JavaScript and CSS Magento is notorious for loading dozens of CSS and JavaScript files. This has a significant impact on rendering and, consequently, on load time and Core Web Vitals. It’s important to load CSS and JavaScript files only where they’re actually used. Additionally, it’s possible to bundle files and defer their loading.
A user-friendly URL structure Without the correct settings, Magento does not provide URLs with an optimal URL structure. For the structure, it is recommended to replace the .html extension with a trailing slash and to build the URLs in a hierarchical manner. Suppose you sell clothing; in that case, the URL structure shown below is easy for users to navigate and helps crawlers understand the structure of your online store: Main category: /children’s clothing/ Subcategory: /children’s-clothing/t-shirts/ Parent category: /clothing/t-shirts/white/
Redirects are not created automatically In practice, we see that URLs in e-commerce environments change regularly for a variety of reasons. By default, Magento does not set up a redirect from the old URL to the new one, which results in manual work or 404 pages (leading to a decline in search visibility). Through Magento’s settings, it’s easy to enable automatic redirects so that users and crawlers are always directed to the correct new URL.