[invisit](https://invisit.dev) · [Docs](https://invisit.dev/docs) · [llms.txt](https://invisit.dev/llms.txt) · [llms-full.txt](https://invisit.dev/llms-full.txt)

# Meta-ExternalFetcher

> Meta-ExternalFetcher is an AI assistant fetcher run by Meta. It fetches individual links at a user's request and supports agentic AI features, including helping AI navigate websites to complete tasks for users.

- Operator: Meta
- Kind: AI assistant fetcher
- robots.txt token: meta-externalfetcher
- Follows robots.txt: partly
- Source: https://developers.facebook.com/docs/sharing/webmasters/web-crawlers (checked 2026-11-05)
- Page: https://invisit.dev/bots/meta-externalfetcher

## What is Meta-ExternalFetcher?

It fetches individual links at a user's request and supports agentic AI features, including helping AI navigate websites to complete tasks for users.

Fetches a page because a person asked an assistant about it, right now. Operators often say robots.txt does not apply to these user-triggered fetches.

Meta says this crawler may bypass robots.txt because the fetches are requested by users. Meta crawlers may cache robots.txt for up to 24 hours.

- [Meta’s documentation](https://developers.facebook.com/docs/sharing/webmasters/web-crawlers)

## Meta-ExternalFetcher user agent

Meta documents these user agents for Meta-ExternalFetcher. Version numbers in it change over time, so match on the name, not the whole string.

```
meta-externalfetcher/1.1 (+/documentation/sharing/webmasters/web-crawlers)
meta-externalfetcher/1.1
```

## Should you block Meta-ExternalFetcher?

Usually allow it. Each fetch is a person asking about your page right now, which is close to a visit. Blocking it in robots.txt may not stop it, since operators often treat these fetches as the person’s own.

## How to block Meta-ExternalFetcher in robots.txt

Add a group for its token to robots.txt at the root of your site. Meta says it follows robots.txt only in part, so this may not stop every fetch.

A bot follows only the group that names it, so it ignores your `User-agent: *` rules once it has a group of its own. Repeat any paths you also want kept from it there.

```
User-agent: meta-externalfetcher
Disallow: /
```

- [Build a robots.txt for AI bots](https://invisit.dev/tools/robots-txt-generator)

## How to verify a request is really Meta-ExternalFetcher

Anyone can send any user agent, so the string alone proves nothing.

Meta publishes no IP ranges or DNS names for Meta-ExternalFetcher that we could find, so there is no reliable way to tell a real Meta-ExternalFetcher from a scraper using its user agent.

## See when Meta-ExternalFetcher reads your site

Bots don’t run JavaScript analytics, so a script tag never sees them. invisit records crawler reads on the server instead, through its server SDK or a Cloudflare Worker, and shows which bots read which pages, next to your visitors.

- [Track crawlers with invisit](https://invisit.dev/docs/server/sdk#crawler-reads)

## Common questions

### What is Meta-ExternalFetcher?

Meta-ExternalFetcher is an AI assistant fetcher run by Meta. It fetches individual links at a user's request and supports agentic AI features, including helping AI navigate websites to complete tasks for users.

### Is Meta-ExternalFetcher safe to block?

Usually allow it. Each fetch is a person asking about your page right now, which is close to a visit. Blocking it in robots.txt may not stop it, since operators often treat these fetches as the person’s own.

### How do I block Meta-ExternalFetcher?

Add “User-agent: meta-externalfetcher” followed by “Disallow: /” to your robots.txt. Meta doesn’t promise to follow it in every case, so block it at your server too if it matters.