What is AdsBot-Google?
AdsBot checks the quality of web pages used in Google Ads. Mediapartners-Google, grouped with it here, visits AdSense and Ad Manager sites to show them relevant ads.
A bot that fits none of the groups above, such as an ad review crawler.
These crawlers ignore the global User-agent: * group, with the ad publisher's permission, so you must name them to restrict them.
AdsBot-Google user agent
Google documents these user agents for AdsBot-Google. Version numbers in it change over time, so match on the name, not the whole string.
AdsBot-Google (+http://www.google.com/adsbot.html)
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; AdsBot-Google-Mobile; +http://www.google.com/mobile/adsbot.html)
Mediapartners-GoogleShould you block AdsBot-Google?
It depends on what it does; read the operator’s page below.
How to block AdsBot-Google in robots.txt
Add a group for its tokens to robots.txt at the root of your site. Google says it follows robots.txt only in part, so this may not stop every fetch.
A bot follows only the group that names it, so it ignores your User-agent: * rules once it has a group of its own. Repeat any paths you also want kept from it there.
User-agent: AdsBot-Google
User-agent: AdsBot-Google-Mobile
User-agent: Mediapartners-Google
Disallow: /How to verify a request is really AdsBot-Google
Anyone can send any user agent, so the string alone proves nothing.
Google publishes the IP ranges AdsBot-Google crawls from. A request that claims to be AdsBot-Google from any other address is not AdsBot-Google.
Or check by reverse DNS: look up the hostname of the request’s IP, confirm it ends in google.com, then look that hostname up again and confirm it points back to the same IP.
See when AdsBot-Google reads your site
Bots don’t run JavaScript analytics, so a script tag never sees them. invisit records crawler reads on the server instead, through its server SDK or a Cloudflare Worker, and shows which bots read which pages, next to your visitors.
Checked against Google’s own documentation on . Something changed? Write to help@invisit.dev.