What is FeedFetcher-Google?
It fetches RSS and Atom feeds for Google News and WebSub, at the request of users of apps and services.
A bot that fits none of the groups above, such as an ad review crawler.
Google says Feedfetcher ignores robots.txt because it acts for human users, and it should not fetch a feed more than about once an hour on average.
FeedFetcher-Google user agent
Google documents this user agent for FeedFetcher-Google. Version numbers in it change over time, so match on the name, not the whole string.
FeedFetcher-Google; (+http://www.google.com/feedfetcher.html)Should you block FeedFetcher-Google?
It depends on what it does; read the operator’s page below.
How to block FeedFetcher-Google in robots.txt
Google says FeedFetcher-Google does not follow robots.txt, so a robots.txt rule won’t stop it. Block it at your server or firewall by user agent, and by IP if Google publishes its ranges.
Google suggests serving a 404, 410 or other error status to the Feedfetcher-Google user agent if you do not want your feeds fetched.
How to verify a request is really FeedFetcher-Google
Anyone can send any user agent, so the string alone proves nothing.
Google publishes the IP ranges FeedFetcher-Google crawls from. A request that claims to be FeedFetcher-Google from any other address is not FeedFetcher-Google.
Or check by reverse DNS: look up the hostname of the request’s IP, confirm it ends in google.com, then look that hostname up again and confirm it points back to the same IP.
See when FeedFetcher-Google reads your site
Bots don’t run JavaScript analytics, so a script tag never sees them. invisit records crawler reads on the server instead, through its server SDK or a Cloudflare Worker, and shows which bots read which pages, next to your visitors.
Checked against Google’s own documentation on . Something changed? Write to help@invisit.dev.