What is OAI-SearchBot?
It crawls pages so they can be shown and linked in ChatGPT's search features. OpenAI says sites that opt out are not shown in ChatGPT search answers, though they can still appear as navigational links.
Indexes pages so an AI search product can cite and link them in answers. Blocking it usually means you stop showing up as a source there.
OpenAI says robots.txt changes can take about 24 hours to apply to search. If both OAI-SearchBot and GPTBot are allowed, OpenAI may use one crawl for both purposes.
OAI-SearchBot user agent
OpenAI documents these user agents for OAI-SearchBot. Version numbers in it change over time, so match on the name, not the whole string.
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; robots.txt; +https://openai.com/searchbotShould you block OAI-SearchBot?
Usually allow it. It is how an AI search product finds pages to cite and link, so blocking it trades away being a source in those answers.
How to block OAI-SearchBot in robots.txt
Add a group for its token to robots.txt at the root of your site.
A bot follows only the group that names it, so it ignores your User-agent: * rules once it has a group of its own. Repeat any paths you also want kept from it there.
User-agent: OAI-SearchBot
Disallow: /How to verify a request is really OAI-SearchBot
Anyone can send any user agent, so the string alone proves nothing.
OpenAI publishes the IP ranges OAI-SearchBot crawls from. A request that claims to be OAI-SearchBot from any other address is not OAI-SearchBot.
See when OAI-SearchBot reads your site
Bots don’t run JavaScript analytics, so a script tag never sees them. invisit records crawler reads on the server instead, through its server SDK or a Cloudflare Worker, and shows which bots read which pages, next to your visitors.
Checked against OpenAI’s own documentation on . Something changed? Write to help@invisit.dev.