See it on a real storeBots, pages and people sent by AI on an example store. No sign-up.AI training, run by Anthropic.
What is ClaudeBot? Anthropic's crawler that collects AI training data. It can't be verified beyond its name.
Run by Anthropic, which documents it.
Purpose: AI training. A visit means your content is being collected, possibly for training.
robots.txt name: ClaudeBot.
Name only: no published list to check a visit against.
At a glance, the facts in one table.
ClaudeBot at a glance. Taken from SenseGap's open bot directory, the same data that sorts every visit in the dashboard.
All bots| ClaudeBot | |
|---|---|
| Operator | Anthropic |
| Purpose | AI training |
| User agent contains | ClaudeBot |
| robots.txt name | ClaudeBot |
| Verification | Name only |
| Documentation | support.claude.com |
Purpose, what it does and why it came.
What ClaudeBot does. SenseGap sorts every visit by purpose. ClaudeBot is in the AI training group.
ClaudeBot is Anthropic's crawler that collects web content that may be used to train AI models.
What a visit means
A visit means your content is being collected, possibly for training. Nobody asked about you; training crawlers read the web in bulk and don't link back.
Training crawls say nothing about what people ask right now. SenseGap keeps them apart from visits made while answering a person, so a crawl spike never looks like demand.
Detection, the header SenseGap matches.
How SenseGap recognizes ClaudeBot. By its User-Agent header. A separate check then decides whether the visit is real.
Every request carries a User-Agent header naming the software that sent it. SenseGap counts a request as ClaudeBot when that header contains ClaudeBot, in any letter case. The header is plain text, so any script can copy it. Matching the name is where recognition starts, not where trust starts.
/ClaudeBot/iVerification, how to tell a real visit from a fake.
ClaudeBot can't be fully verified. SenseGap knows of no published IP ranges or DNS names for it, so the name is all there is.
How we verify botsNot beyond the user agent. SenseGap knows of no published IP ranges or DNS names for ClaudeBot, so anyone can send its name. SenseGap marks these visits Unverified rather than calling them real.
robots.txt, rules you can copy.
Block or allow ClaudeBot in robots.txt. Its robots.txt name is ClaudeBot. Paste a group into the robots.txt at your site's root.
User-agent: ClaudeBot
Disallow: /User-agent: ClaudeBot
Allow: /User-agent: ClaudeBot
Disallow: /private/One rule catches people out. Under the robots.txt standard (RFC 9309), a crawler that finds a group naming it follows only that group and ignores your User-agent: * rules. If you add a ClaudeBot group, repeat in it anything from the catch-all group you still want applied.
robots.txt is a request, not a lock. To enforce a block, match the User-Agent header at your server or CDN. In nginx:
# nginx, inside a server block
if ($http_user_agent ~* "ClaudeBot") {
return 403;
}SenseGap reads your robots.txt and shows, bot by bot, what you allow and which bots fetched pages you disallowed. See what you get.
Trade-off, what blocking changes.
Should you block ClaudeBot? It depends on what you want from Anthropic. Here is what each choice means.
If you allow it
Future models may learn from your content, so they know your products, prices and wording.
If you block it
New content stops reaching that company's training data. It doesn't remove what was already collected, and because training crawlers don't send visitors, it costs no clicks directly. Anthropic runs separate bots for training, search and user requests, so blocking ClaudeBot does not stop Claude-User or Claude-SearchBot.
Anthropic, every bot it runs.
More bots from Anthropic. Anthropic runs 4 bots in the directory, each with its own job.
AI training, other bots with the same job.
Other AI training crawlers. Same purpose, different companies.
Browse the directoryFAQ, ClaudeBot in short answers.
ClaudeBot, answered plainly
What is ClaudeBot?
ClaudeBot is Anthropic's crawler that collects web content that may be used to train AI models.
Who runs ClaudeBot?
Anthropic. Anthropic documents it at https://support.claude.com/en/articles/8896518.
What does a visit from ClaudeBot mean?
A visit means your content is being collected, possibly for training. Nobody asked about you; training crawlers read the web in bulk and don't link back.
What user agent does ClaudeBot use?
Its User-Agent header contains "ClaudeBot". SenseGap matches it regardless of letter case.
How do I block ClaudeBot?
Add "User-agent: ClaudeBot" followed by "Disallow: /" to your robots.txt. robots.txt is a request, not a lock; to enforce it, block the user agent at your server or CDN.
Can I verify that a visit is really ClaudeBot?
Not beyond the user agent. SenseGap knows of no published IP ranges or DNS names for ClaudeBot, so anyone can send its name. SenseGap marks these visits Unverified rather than calling them real.
Does Anthropic publish IP ranges for ClaudeBot?
No. SenseGap knows of no published list of addresses for ClaudeBot.
What happens if I block ClaudeBot?
New content stops reaching that company's training data. It doesn't remove what was already collected, and because training crawlers don't send visitors, it costs no clicks directly. Anthropic runs separate bots for training, search and user requests, so blocking ClaudeBot does not stop Claude-User or Claude-SearchBot.