Fix “Desktop page not crawlable due to robots.txt” in Merchant Center

Short answer. Merchant Center disapproves a product with “Desktop page not crawlable due to robots.txt” (or the mobile equivalent) when the robots.txt on your store disallows Googlebot from the product URL. Allow Googlebot on your product paths in robots.txt; Google re-reads the file on its own and re-crawls the pages, and a feed re-fetch prompts it sooner.

What Merchant Center means by “Desktop page not crawlable due to robots.txt”

Before crawling a product page to verify price and availability, Google reads your robots.txt. If it disallows Googlebot from the product URL, Google obeys, cannot verify the product, and disapproves it. The Content API’s product issues list has both forms: landing_page_roboted, “Desktop page not crawlable due to robots.txt”, and mobile_landing_page_roboted for the smartphone crawl; the fix for both is “update your robots.txt file”. Google’s robots.txt help page gives the two lines, and its page on unavailable landing pages covers the neighbouring issue where the page is allowed but does not load.

The same file can block images and pages separately, so a store can have this issue and its image twin at once; the fix is the same file.

Why it happens

• A staging robots.txt (Disallow: /) shipped to production at launch.
• Product paths disallowed to keep filtered or parameterised URLs out of search, with a pattern that also catches the canonical product pages.
• Query strings blocked (Disallow: /*?) on a store whose variant links carry a query.
• A rule for Googlebot specifically, or for the smartphone agent alone.

How to fix it

In Shopify

Shopify’s default robots.txt allows product pages. If you edited robots.txt.liquid, compare it with the default and remove any rule covering /products/ or query strings on product URLs. Shopify’s own variant links use ?variant=, so a blanket rule on ? blocks them.

In WooCommerce

Open https://your-domain/robots.txt. Remove any Disallow covering /product/ (or your product base) and any rule on query strings, since variation links carry attribute parameters. An SEO plugin may be generating the file; edit it there. Then test one product URL with Search Console’s robots.txt report.

In a CSV or XML feed

Test each distinct product URL pattern from the feed against your robots.txt (Search Console’s robots.txt report does this for hosts you own). Allow Googlebot on those paths, as Google’s page shows: User-agent: Googlebot then Disallow: with nothing after it, or simply remove the offending rules. Re-fetch the feed to prompt a re-crawl.

How long until Merchant Center clears it

Google re-reads robots.txt on its own and re-crawls the pages; Search Console can request an immediate re-read of the file. Products are re-approved once a crawl succeeds, typically within 24–72 hours of the fix and fetch.

What FeedRobin does about it

The free feed check opens a sample of product pages from our servers, not Google’s, so a rule aimed at Googlebot does not stop it and this issue is not something it reports. It does show whether the sampled pages loaded and what price they carried, which tells you the pages themselves work. For connected stores the fix is the same robots.txt on your storefront.

Sources

• How to fix: Robots.txt error — Google Merchant Center Help
• Product issues (Content API for Shopping): the issue codes and the titles Merchant Center shows — Google Merchant Center Help
• How to fix: Unavailable desktop landing page — Google Merchant Center Help