What is DotBot?
Moz's web crawler, which gathers data for the Moz Link Index behind Link Explorer and the Moz Links API. Moz's other crawler, rogerbot, crawls sites for Moz Pro Site Crawl.
Crawls for an SEO product: backlink indexes, site audits, rank data. It sends no visitors; blocking it is common and harmless.
Moz says dotbot only reads robots.txt the first time it meets a site in a new index crawl, so changes may wait until the next crawl. Moz has no fixed IP range for rogerbot.
Should you block DotBot?
Block it freely if the load bothers you. It sends no visitors; the only cost is that the tool’s users, your competitors included, see less of your site.
How to block DotBot in robots.txt
Add a group for its tokens to robots.txt at the root of your site.
A bot follows only the group that names it, so it ignores your User-agent: * rules once it has a group of its own. Repeat any paths you also want kept from it there.
User-agent: dotbot
User-agent: rogerbot
Disallow: /How to verify a request is really DotBot
Anyone can send any user agent, so the string alone proves nothing.
Moz publishes no IP ranges or DNS names for DotBot that we could find, so there is no reliable way to tell a real DotBot from a scraper using its user agent.
See when DotBot reads your site
Bots don’t run JavaScript analytics, so a script tag never sees them. invisit records crawler reads on the server instead, through its server SDK or a Cloudflare Worker, and shows which bots read which pages, next to your visitors.
Checked against Moz’s own documentation on . Something changed? Write to help@invisit.dev.