What is AhrefsBot?
It powers Ahrefs' marketing data platform and Yep, Ahrefs' search engine. A second bot, AhrefsSiteAudit, powers Ahrefs' Site Audit tool.
Crawls for an SEO product: backlink indexes, site audits, rank data. It sends no visitors; blocking it is common and harmless.
Only a site's verified owner can let AhrefsSiteAudit ignore robots.txt on that site. Crawl-delay is followed for HTML pages but not for assets loaded while rendering JavaScript.
AhrefsBot user agent
Ahrefs documents these user agents for AhrefsBot. Version numbers in it change over time, so match on the name, not the whole string.
Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/)
Mozilla/5.0 (compatible; AhrefsSiteAudit/6.1; +http://ahrefs.com/robot/site-audit)Should you block AhrefsBot?
Block it freely if the load bothers you. It sends no visitors; the only cost is that the tool’s users, your competitors included, see less of your site.
How to block AhrefsBot in robots.txt
Add a group for its tokens to robots.txt at the root of your site.
A bot follows only the group that names it, so it ignores your User-agent: * rules once it has a group of its own. Repeat any paths you also want kept from it there.
User-agent: AhrefsBot
User-agent: AhrefsSiteAudit
Disallow: /How to verify a request is really AhrefsBot
Anyone can send any user agent, so the string alone proves nothing.
Ahrefs publishes the IP ranges AhrefsBot crawls from. A request that claims to be AhrefsBot from any other address is not AhrefsBot.
Or check by reverse DNS: look up the hostname of the request’s IP, confirm it ends in ahrefs.com or ahrefs.net, then look that hostname up again and confirm it points back to the same IP.
See when AhrefsBot reads your site
Bots don’t run JavaScript analytics, so a script tag never sees them. invisit records crawler reads on the server instead, through its server SDK or a Cloudflare Worker, and shows which bots read which pages, next to your visitors.
Checked against Ahrefs’s own documentation on . Something changed? Write to help@invisit.dev.