What is MJ12bot?
Majestic's distributed crawler, which maps links between websites for Majestic's backlink index.
Crawls for an SEO product: backlink indexes, site audits, rank data. It sends no visitors; blocking it is common and harmless.
Majestic says not to block it by IP because it is a distributed crawler. If it cannot fetch robots.txt it assumes it may crawl, though some failures such as 403 count as a full disallow.
Should you block MJ12bot?
Block it freely if the load bothers you. It sends no visitors; the only cost is that the tool’s users, your competitors included, see less of your site.
How to block MJ12bot in robots.txt
Add a group for its token to robots.txt at the root of your site.
A bot follows only the group that names it, so it ignores your User-agent: * rules once it has a group of its own. Repeat any paths you also want kept from it there.
Majestic can send a pre-arranged secret string in a CRAWLER-IDENT header or the user agent so you can confirm requests come from its network; ask bot@majestic12.co.uk.
User-agent: MJ12bot
Disallow: /How to verify a request is really MJ12bot
Anyone can send any user agent, so the string alone proves nothing.
Majestic publishes no IP ranges or DNS names for MJ12bot that we could find, so there is no reliable way to tell a real MJ12bot from a scraper using its user agent.
See when MJ12bot reads your site
Bots don’t run JavaScript analytics, so a script tag never sees them. invisit records crawler reads on the server instead, through its server SDK or a Cloudflare Worker, and shows which bots read which pages, next to your visitors.
Checked against Majestic’s own documentation on . Something changed? Write to help@invisit.dev.