What is ClaudeBot?
It collects public web content that may be used to train Anthropic's generative AI models. Blocking it signals that your site's future content should be left out of training data.
Collects pages to train language models. Blocking it keeps your pages out of future training data, and does not remove you from AI search answers.
Add the rule on every subdomain you want excluded. Older names such as anthropic-ai and Claude-Web do not appear on Anthropic's current crawler page.
Should you block ClaudeBot?
Block it if you don’t want your pages used to train models; nothing else changes. Allow it if you’d rather future models know your product, docs or writing.
How to block ClaudeBot in robots.txt
Add a group for its token to robots.txt at the root of your site.
A bot follows only the group that names it, so it ignores your User-agent: * rules once it has a group of its own. Repeat any paths you also want kept from it there.
Anthropic supports Crawl-delay and, since October 30, 2026, honors the IETF aipref 'no AI training' preference signal; a robots.txt rule that names ClaudeBot takes priority over that signal.
User-agent: ClaudeBot
Disallow: /How to verify a request is really ClaudeBot
Anyone can send any user agent, so the string alone proves nothing.
Anthropic publishes the IP ranges ClaudeBot crawls from. A request that claims to be ClaudeBot from any other address is not ClaudeBot.
See when ClaudeBot reads your site
Bots don’t run JavaScript analytics, so a script tag never sees them. invisit records crawler reads on the server instead, through its server SDK or a Cloudflare Worker, and shows which bots read which pages, next to your visitors.
Checked against Anthropic’s own documentation on . Something changed? Write to help@invisit.dev.