Bot directory
Every bot we recognize, and how we know it's real.
55bots and AI agents
34companies
26can be verified beyond their name
Answering a person6
| Bot | Company | How we verify | robots.txt name | Docs |
|---|
| ChatGPT-User | OpenAI | IP ranges | ChatGPT-User | Docs |
| Claude-User | Anthropic | Name only | Claude-User | Docs |
| Perplexity-User | Perplexity | IP ranges | Perplexity-User | Docs |
| Google-NotebookLM | Google | IP ranges + Reverse DNS | Google-NotebookLM | |
| MistralAI-User | Mistral AI | IP ranges | MistralAI-User | |
| Meta-ExternalFetcher | Meta | Name only | Meta-ExternalFetcher | |
AI search index8
| Bot | Company | How we verify | robots.txt name | Docs |
|---|
| OAI-SearchBot | OpenAI | IP ranges | OAI-SearchBot | Docs |
| Claude-SearchBot | Anthropic | Name only | Claude-SearchBot | Docs |
| PerplexityBot | Perplexity | IP ranges | PerplexityBot | Docs |
| Google-CloudVertexBot | Google | IP ranges + Reverse DNS | Google-CloudVertexBot | |
| DuckAssistBot | DuckDuckGo | IP ranges | DuckAssistBot | |
| Amazonbot | Amazon | Reverse DNS | Amazonbot | |
| YouBot | You.com | Name only | YouBot | |
| ExaSearchBot | Exa | Name only | ExaSearchBot | Docs |
AI training14
| Bot | Company | How we verify | robots.txt name | Docs |
|---|
| GPTBot | OpenAI | IP ranges | GPTBot | Docs |
| ClaudeBot | Anthropic | Name only | ClaudeBot | Docs |
| anthropic-ai / Claude-Web | Anthropic | Name only | – | |
| CCBot | Common Crawl | IP ranges | CCBot | |
| Meta-ExternalAgent | Meta | Name only | Meta-ExternalAgent | |
| FacebookBot | Meta | Name only | FacebookBot | |
| Bytespider | ByteDance | Name only | Bytespider | |
| cohere-ai | Cohere | Name only | cohere-ai | |
| AI2Bot | Allen Institute for AI | Name only | AI2Bot | |
| Diffbot | Diffbot | Name only | Diffbot | |
| Timpibot | Timpi | Name only | Timpibot | |
| ImagesiftBot | Hive | Name only | ImagesiftBot | |
| Omgilibot | Webz.io | Name only | Omgilibot | |
| PanguBot | Huawei | Name only | PanguBot | |
AI agents1
| Bot | Company | How we verify | robots.txt name | Docs |
|---|
| ChatGPT agent | OpenAI | Signature | – | |
Search engines8
| Bot | Company | How we verify | robots.txt name | Docs |
|---|
| Storebot-Google | Google | IP ranges + Reverse DNS | Storebot-Google | |
| Googlebot | Google | IP ranges + Reverse DNS | Googlebot | Docs |
| Bingbot | Microsoft | IP ranges + Reverse DNS | Bingbot | |
| Applebot | Apple | IP ranges + Reverse DNS | Applebot | Docs |
| DuckDuckBot | DuckDuckGo | IP ranges | DuckDuckBot | |
| YandexBot | Yandex | Reverse DNS | YandexBot | |
| Baiduspider | Baidu | Reverse DNS | Baiduspider | |
| PetalBot | Huawei | Reverse DNS | PetalBot | |
SEO tools7
| Bot | Company | How we verify | robots.txt name | Docs |
|---|
| Google-InspectionTool | Google | IP ranges + Reverse DNS | Google-InspectionTool | |
| AhrefsBot | Ahrefs | Reverse DNS | AhrefsBot | |
| SemrushBot | Semrush | Reverse DNS | SemrushBot | |
| MJ12bot | Majestic | Name only | MJ12bot | |
| DotBot | Moz | Name only | DotBot | |
| DataForSeoBot | DataForSEO | Name only | DataForSeoBot | |
| Screaming Frog | Screaming Frog | Name only | – | |
Previews, monitors and others11
| Bot | Company | How we verify | robots.txt name | Docs |
|---|
| GoogleOther | Google | IP ranges + Reverse DNS | GoogleOther | |
| AdsBot-Google | Google | IP ranges + Reverse DNS | AdsBot-Google | |
| Google user-triggered fetcher | Google | IP ranges + Reverse DNS | – | |
| facebookexternalhit | Meta | Name only | facebookexternalhit | |
| Twitterbot | X | Name only | Twitterbot | |
| LinkedInBot | LinkedIn | Name only | LinkedInBot | |
| Slackbot | Slack | Name only | Slackbot | |
| Discordbot | Discord | Name only | Discordbot | |
| WhatsApp | Meta | Name only | WhatsApp | |
| TelegramBot | Telegram | Name only | TelegramBot | |
| Uptime monitor | Various | Name only | – | |
See which of these visit your site.