[invisit](https://invisit.dev) · [Docs](https://invisit.dev/docs) · [llms.txt](https://invisit.dev/llms.txt) · [llms-full.txt](https://invisit.dev/llms-full.txt)

# Twitterbot

> Twitterbot is a link preview fetcher run by X. X's crawler that reads Cards markup on linked pages to build preview cards in posts.

- Operator: X
- Kind: Link preview fetcher
- robots.txt token: Twitterbot
- Follows robots.txt: yes
- Source: https://web.archive.org/web/2023/https://developer.twitter.com/en/docs/twitter-for-websites/cards/guides/getting-started (checked 2026-11-05)
- Page: https://invisit.dev/bots/twitterbot

## What is Twitterbot?

X's crawler that reads Cards markup on linked pages to build preview cards in posts.

Fetches a page when someone shares its link, to draw the preview card. Blocking it breaks your previews in that app.

These facts come from X's archived 2023 developer docs; the page has since been removed. Those docs said blocked pages get no card, blocked images get no thumbnail, and cards are cached for 7 days.

- [X’s documentation](https://web.archive.org/web/2023/https://developer.twitter.com/en/docs/twitter-for-websites/cards/guides/getting-started)

## Twitterbot user agent

X documents this user agent for Twitterbot. Version numbers in it change over time, so match on the name, not the whole string.

```
Twitterbot/1.0
```

## Should you block Twitterbot?

Allow it. Blocking it breaks the preview card when someone shares your link.

## How to block Twitterbot in robots.txt

Add a group for its token to robots.txt at the root of your site.

A bot follows only the group that names it, so it ignores your `User-agent: *` rules once it has a group of its own. Repeat any paths you also want kept from it there.

```
User-agent: Twitterbot
Disallow: /
```

- [Build a robots.txt for AI bots](https://invisit.dev/tools/robots-txt-generator)

## How to verify a request is really Twitterbot

Anyone can send any user agent, so the string alone proves nothing.

X publishes no IP ranges or DNS names for Twitterbot that we could find, so there is no reliable way to tell a real Twitterbot from a scraper using its user agent.

## See when Twitterbot reads your site

Bots don’t run JavaScript analytics, so a script tag never sees them. invisit records crawler reads on the server instead, through its server SDK or a Cloudflare Worker, and shows which bots read which pages, next to your visitors.

- [Track crawlers with invisit](https://invisit.dev/docs/server/sdk#crawler-reads)

## Common questions

### What is Twitterbot?

Twitterbot is a link preview fetcher run by X. X's crawler that reads Cards markup on linked pages to build preview cards in posts.

### Is Twitterbot safe to block?

Allow it. Blocking it breaks the preview card when someone shares your link.

### How do I block Twitterbot?

Add “User-agent: Twitterbot” followed by “Disallow: /” to your robots.txt.