invisit
DocsPricingStart free
Free tools

AI visibility checker

Enter a page and see what AI crawlers get: what robots.txt tells each AI bot, whether the page answers them like it answers a browser, and whether the text is there without JavaScript.

.mdllms.txt

Enter a page and we fetch it the way AI crawlers would: robots.txt, the page as four AI bots and as a browser, and the files they look for. Takes a few seconds.

What it checks

The checker fetches the page the way a crawler would, from our servers, with nothing run in a browser:

  • robots.txt, parsed the way RFC 9309 says crawlers should: a bot obeys the group that names it, and falls back to User-agent: * only if none does. Then, for each AI search bot, assistant and training crawler we track, plus Googlebot and Bingbot for comparison, whether it may fetch your home page and this page.
  • The page as four AI crawlers. We request it as GPTBot, OAI-SearchBot, ClaudeBot and PerplexityBot and compare their answers with a browser’s. A 403 or a challenge page for a bot while a browser gets 200 means something between your server and the bot is turning it away.
  • Text without JavaScript. Most AI crawlers read the HTML as it arrives. If the content only appears after scripts run, they see an empty page.
  • noindex, nofollow and noai, in meta tags and the X-Robots-Tag header.
  • The sitemap, the canonical URL and JSON-LD, which help any crawler find the right page and understand what it is.
  • Optional extras: a Content-Signal line in robots.txt, /llms.txt, and a Markdown version of the page. They are shown but not counted, since no crawler is obliged to read them.

The usual reasons a page is invisible

A bot setting at the CDN. This is the most common one we see. Cloudflare’s AI crawler controls, Vercel’s bot protection and similar firewalls can block AI crawlers for the whole site, and robots.txt never gets a say. Check your CDN’s bot settings first when the crawler fetches fail.

One line in robots.txt. A Disallow: / under User-agent: * keeps out every bot that has no group of its own. And a group for one bot replaces the * rules for that bot entirely, so a bot-specific group with no rules lets that bot in everywhere.

A page built only in the browser. Single-page apps often send an empty <div id="root"> and fill it with JavaScript. Server rendering or static generation puts the text in the HTML, which is what crawlers read.

noindex left over from staging. It keeps the page out of search indexes, and AI search built on those indexes can’t cite it either.

What this check can’t see

It tests one page, from one place, with each crawler’s user agent but not its IP addresses. It doesn’t run JavaScript, log in, or accept a cookie banner. It can tell you that nothing on your side blocks a crawler; it can’t tell you what an AI product does with the page afterwards.

To see which AI crawlers actually read your pages, and how many visitors come back from AI answers, invisit records crawler reads next to your visits.

Common questions

If every check passes, will ChatGPT cite my page?

Not necessarily. Passing means nothing on your side stops a crawler from fetching and reading the page. Whether an assistant uses or cites it depends on its own index, ranking and the question someone asks, which no outside test can see.

Why does the page load for me but fail for GPTBot here?

Usually a CDN or firewall rule. Cloudflare, Vercel, Akamai and others can block AI crawlers with one setting, and some block anything that calls itself a bot. The result names the vendor when the block page gives it away.

Is a request with GPTBot’s user agent the same as GPTBot?

No. Real AI crawlers come from published IP ranges, and many sites check those too. A site can allow the real GPTBot and refuse this test, or the other way round. Treat a block here as a strong hint, and confirm in your CDN’s bot settings or logs.

Do I need llms.txt?

No. It is a proposed file that lists your key pages for language models. No major AI crawler has said it reads it, so the check is shown as optional and never counts against you.

Should I block AI training bots?

That is your call, which is why the report lists training crawlers as a note, not a failure. Blocking them keeps your pages out of future training data and does not remove you from AI search or assistant answers, which use different bots.

What do you keep from my check?

The report is cached for ten minutes so a shared link doesn’t refetch the site, then dropped. This page counts its own visits with invisit like the rest of the site, and the address of a visit includes the ?url= of a shared link.

More free tools