Merchant Center fetches your product feed on a schedule nobody watches. When that fetch fails, nothing in your store changes and nothing shouts: the products already uploaded simply age out, and the catalogue thins over a few days. By the time anyone notices the drop in traffic, the last thing they suspect is a feed that stopped downloading.
This guide covers every reason Google documents for a failed fetch — and one it never reports as a failure at all, because the download succeeds.
What a scheduled fetch actually requires
Google's requirement is narrower than it sounds: the URL you register must return the file itself, to an anonymous crawler, every time. Its own troubleshooting page gives two causes for automatic uploads failing: the URL is “roboted or doesn't have permission settings for Merchant Center to download the data feed”, and the URL “isn't working or redirects to a page that doesn't contain the data feed”.
There are hard limits too: files fetched by schedule, uploaded directly or sent over SFTP must stay under 4 GB, and a compressed file under 500 MB.
Cause 1: robots.txt blocks the feed URL
The most common one, and the least obvious, because the feed opens perfectly in your own browser. If robots.txt disallows the path your feed sits on — or disallows everything for non-Google agents while the fetch comes from a different crawler than you expect — the download never happens. Google's robots.txt fix asks you to allow both Googlebot, which crawls your landing pages, and Googlebot-image, which crawls your images, across the whole site.
Worth separating two things that share a name: the feed file being blocked stops the upload completely, while your product pages being blocked lets the feed in but disapproves items for uncheckable landing pages. Different symptoms, same file to edit.
Cause 2: the URL returns a page, not a file
A feed URL can answer 200 OK and still be useless. The usual shapes:
A login or consent wall. Your session has a cookie; Google's fetch has nothing. What comes back is the login form, which is valid HTML and contains no products.
A redirect to a landing page. Store platforms often rewrite unknown paths to the homepage instead of returning 404. Google follows the redirect, receives a page, and reports a data source it cannot read.
A “your export is being generated” page. Plugins that build the file on demand sometimes answer immediately with a placeholder and finish in the background. The first fetch gets the placeholder.
The check is the response header, not the browser view: Content-Type should be application/xml, text/xml or text/csv — never text/html.
Cause 3: SFTP errors, named exactly
If you push instead of letting Google pull, the same page names the errors: “Server not found” (check the connection to partnerupload.google.com on port 19321), “Username or password incorrect”, “Filename not allowed” (remove spaces; the filename must match the registered one exactly), “SFTP connection reset by peer” and “Authentication failed”.
The failure Google never calls a failure
Now the one that costs the most, because no error is ever raised: the fetch succeeds, the file is well-formed, and it contains nothing.
We hit this on a WooCommerce test store dressed like a real one — Storefront, an SEO plugin, and a page cache, which is an ordinary thing for a shop to have. The feed endpoint streams its XML rather than building the whole document in memory, so a catalogue of any size fits in a page view's limits. The cache plugin wraps every request in an output buffer and treats whatever is in that buffer at the end as “the page”. With a streamed response, what was left in the buffer was the last chunk — eighteen bytes, </channel></rss> — and that was stored and served to every fetch afterwards.
Read that as a machine would: it is a well-formed XML document containing zero products. Merchant Center doesn't report a broken feed, it reports an empty one — or simply stops showing your items. Nothing is logged, the URL keeps answering 200, and the catalogue disappears from every channel at once, days after the last change anybody made.
Any component that opens an output buffer does this to a streamed response: a full-page cache, an HTML minifier, a debug toolbar. The cache plugin is simply the most common one to have installed.
The 60-second check on your own store
Run both commands twice. The first request is often the honest one; the second is what Google gets.
curl -sI "https://yourstore.com/your-feed-url" curl -s "https://yourstore.com/your-feed-url" | wc -c
Content-Type: text/html means you are serving a page, not a feed. A second response dramatically smaller than the first means something cached a fragment of a streamed file. A redirect chain in the headers means Google is being sent somewhere else entirely.
If you maintain the feed endpoint yourself
On WordPress, two lines fix the whole class — the constant tells caching plugins to leave the response alone, and the loop closes any buffer one of them has already opened:
if ( ! defined( 'DONOTCACHEPAGE' ) ) {
define( 'DONOTCACHEPAGE', true ); // honoured by WP Super Cache, W3 Total Cache, LiteSpeed, WP Rocket
}
while ( ob_get_level() > 0 ) {
ob_end_clean();
}
nocache_headers();The constant alone was not enough in our case: a buffer was already open before our code ran, so the second half is the part that actually fixed it. If you don't maintain the endpoint — you use someone's feed plugin — exclude the feed URL in your cache plugin's settings, then re-run the check above.
How FeedRobin removes the failure mode
FeedRobin serves the feed when the channel asks for it, rather than writing a file on a schedule and hoping it is still there — so there is no stale export to go missing, and the URL you register never redirects. Our WooCommerce connector sets DONOTCACHEPAGE and closes open buffers before it writes a byte, because that is the bug above, found on a real install and fixed. And the free feed check fetches your feed URL the way Google does — anonymous, no cookies — so an empty or HTML response shows up as a finding instead of a silent disappearance.
If your products have already thinned out, work through the four-step diagnostic first: a feed that stopped arriving and a feed that arrives with prices Google disagrees with look identical from the outside.
Sources
Google Merchant Center Help: Troubleshoot my data source (automatic upload failures, the named SFTP errors, the 4 GB and 500 MB limits) and How to fix: Robots.txt error. The empty-feed case is our own, found on a WooCommerce install running WP Super Cache in September 2026 and fixed in our connector; the two commands above are the check we used to confirm it.