What is Meta-ExternalAgent?
It crawls the web for uses such as training foundation AI models or improving products by indexing content directly.
Collects pages to train language models. Blocking it keeps your pages out of future training data, and does not remove you from AI search answers.
Meta crawlers may cache robots.txt for up to 24 hours. FacebookBot, an older Meta crawler name, is not on Meta's current crawler page.
Meta-ExternalAgent user agent
Meta documents these user agents for Meta-ExternalAgent. Version numbers in it change over time, so match on the name, not the whole string.
meta-externalagent/1.1 (+/documentation/sharing/webmasters/web-crawlers)
meta-externalagent/1.1Should you block Meta-ExternalAgent?
Block it if you don’t want your pages used to train models; nothing else changes. Allow it if you’d rather future models know your product, docs or writing.
How to block Meta-ExternalAgent in robots.txt
Add a group for its token to robots.txt at the root of your site.
A bot follows only the group that names it, so it ignores your User-agent: * rules once it has a group of its own. Repeat any paths you also want kept from it there.
Meta says it uses robots.txt rather than non-standard formats such as NoAI tags to read site preferences.
User-agent: meta-externalagent
Disallow: /How to verify a request is really Meta-ExternalAgent
Anyone can send any user agent, so the string alone proves nothing.
Meta publishes no IP ranges or DNS names for Meta-ExternalAgent that we could find, so there is no reliable way to tell a real Meta-ExternalAgent from a scraper using its user agent.
See when Meta-ExternalAgent reads your site
Bots don’t run JavaScript analytics, so a script tag never sees them. invisit records crawler reads on the server instead, through its server SDK or a Cloudflare Worker, and shows which bots read which pages, next to your visitors.
Checked against Meta’s own documentation on . Something changed? Write to help@invisit.dev.