ClaudeBot

ClaudeBot is the user agent Anthropic uses when crawling the web for Claude. Like other AI crawlers, it can be allowed or blocked via robots.txt.

User agent

User-agent: ClaudeBot

Related Anthropic crawlers

Anthropic also operates additional user agents for different surfaces:

• ClaudeBot — primary crawler
• Claude-Web / anthropic-ai — historical or alternate user agents you may see in logs
• claude-user — on-demand fetches when a Claude user clicks a link

Why ClaudeBot matters for GEO

Claude is increasingly used by knowledge workers and developers for research, drafting, and decision support. Blocking ClaudeBot means your content cannot be retrieved when Claude grounds answers with live web search, which lowers your answer share inside Claude-driven workflows.

Example directive

User-agent: ClaudeBot
Allow: /

User-agent: anthropic-ai
Allow: /