[invisit](https://invisit.dev) · [Docs](https://invisit.dev/docs) · [llms.txt](https://invisit.dev/llms.txt) · [llms-full.txt](https://invisit.dev/llms-full.txt)

# Claude-SearchBot

> Claude-SearchBot is an AI search crawler run by Anthropic. It crawls the web to improve the relevance and accuracy of search results for Claude users.

- Operator: Anthropic
- Kind: AI search crawler
- robots.txt token: Claude-SearchBot
- Follows robots.txt: yes
- Source: https://support.claude.com/en/articles/8896518 (checked 2026-11-05)
- Page: https://invisit.dev/bots/claude-searchbot

## What is Claude-SearchBot?

It crawls the web to improve the relevance and accuracy of search results for Claude users.

Indexes pages so an AI search product can cite and link them in answers. Blocking it usually means you stop showing up as a source there.

Anthropic says blocking Claude-SearchBot may reduce your site's visibility and accuracy in Claude's search results.

- [Anthropic’s documentation](https://support.claude.com/en/articles/8896518)

## Should you block Claude-SearchBot?

Usually allow it. It is how an AI search product finds pages to cite and link, so blocking it trades away being a source in those answers.

## How to block Claude-SearchBot in robots.txt

Add a group for its token to robots.txt at the root of your site.

A bot follows only the group that names it, so it ignores your `User-agent: *` rules once it has a group of its own. Repeat any paths you also want kept from it there.

```
User-agent: Claude-SearchBot
Disallow: /
```

- [Build a robots.txt for AI bots](https://invisit.dev/tools/robots-txt-generator)

## How to verify a request is really Claude-SearchBot

Anyone can send any user agent, so the string alone proves nothing.

Anthropic publishes the IP ranges Claude-SearchBot crawls from. A request that claims to be Claude-SearchBot from any other address is not Claude-SearchBot.

- [Claude-SearchBot IP ranges](https://claude.com/crawling/bots.json)

## See when Claude-SearchBot reads your site

Bots don’t run JavaScript analytics, so a script tag never sees them. invisit records crawler reads on the server instead, through its server SDK or a Cloudflare Worker, and shows which bots read which pages, next to your visitors.

- [Track crawlers with invisit](https://invisit.dev/docs/server/sdk#crawler-reads)

## Common questions

### What is Claude-SearchBot?

Claude-SearchBot is an AI search crawler run by Anthropic. It crawls the web to improve the relevance and accuracy of search results for Claude users.

### Is Claude-SearchBot safe to block?

Usually allow it. It is how an AI search product finds pages to cite and link, so blocking it trades away being a source in those answers.

### How do I block Claude-SearchBot?

Add “User-agent: Claude-SearchBot” followed by “Disallow: /” to your robots.txt.