Skip to main content
Back to report

robots.txt allows ClaudeBot

GEOWeight 7.0% (This check's share of the GEO score)Score 100

What this is

Whether your robots.txt (the file that tells crawlers which parts of the site they may visit) blocks ClaudeBot, the crawler Anthropic uses to collect data for training Claude.

Why it matters

Blocking ClaudeBot mainly decides whether your content is used to train Claude. Claude uses other crawlers when it searches the web, so this has little effect on Claude finding your pages in a search. Allowing it is your choice.

What counts as passing

Passes unless robots.txt has a section for ClaudeBot with “Disallow: /” (block the whole site); not scored if the site has no robots.txt.

Evidence

Crawler nameClaudeBot
CrawlerClaudeBot (Anthropic)
BlockedNo
Has its own rulesNo
Raw data

Check ID geo.robots.allowsClaudeBot

{
  "bot": "ClaudeBot",
  "label": "ClaudeBot (Anthropic)",
  "blocked": false,
  "matched": false
}

Suggestion

Pro plan only

Allow ClaudeBot (Anthropic)

robots.txt currently blocks ClaudeBot, preventing AI answer engines from indexing the site.

Before / After

Current
User-agent: ClaudeBot
Disallow: /
Suggested
User-agent: ClaudeBot
Disallow: