What is Claude-User?
It fetches pages when a Claude user asks a question that needs content from the web.
Fetches a page because a person asked an assistant about it, right now. Operators often say robots.txt does not apply to these user-triggered fetches.
Anthropic says blocking Claude-User stops Claude retrieving your pages for user queries, which may reduce your visibility in user-directed search. It warns that blocking its IP addresses instead of using robots.txt may not give a reliable opt-out.
Should you block Claude-User?
Usually allow it. Each fetch is a person asking about your page right now, which is close to a visit. Blocking it in robots.txt may not stop it, since operators often treat these fetches as the person’s own.
How to block Claude-User in robots.txt
Add a group for its token to robots.txt at the root of your site.
A bot follows only the group that names it, so it ignores your User-agent: * rules once it has a group of its own. Repeat any paths you also want kept from it there.
User-agent: Claude-User
Disallow: /How to verify a request is really Claude-User
Anyone can send any user agent, so the string alone proves nothing.
Anthropic publishes the IP ranges Claude-User crawls from. A request that claims to be Claude-User from any other address is not Claude-User.
See when Claude-User reads your site
Bots don’t run JavaScript analytics, so a script tag never sees them. invisit records crawler reads on the server instead, through its server SDK or a Cloudflare Worker, and shows which bots read which pages, next to your visitors.
Checked against Anthropic’s own documentation on . Something changed? Write to help@invisit.dev.