Aller au contenu

Cette traduction de l'interface est un brouillon qui n'a pas été relu par une personne de langue maternelle. L'anglais est la version source. English

Anthropic documents ClaudeBot training opt-out via robots.txt

7 avr. 2026 · Collecté automatiquement à partir de sources publiques. Non vérifié par une personne.

Ce contenu n'est disponible que dans sa langue d'origine (anglais).

What happened

Anthropic publishes help-center documentation describing the three web robots it operates and how site owners can control them. The training-relevant control: blocking ClaudeBot in robots.txt "signals that the site's future materials should be excluded from our AI model training datasets." The page separately documents Claude-User (fetches pages when a Claude user asks a question) and Claude-SearchBot (indexes content for search-result quality), each with its own opt-out trade-off. Anthropic also supports the non-standard Crawl-delay directive to slow crawls, and warns that blocking by IP address is not a reliable substitute for robots.txt. The page is dated 7 April 2026.

Why it may matter to you

For a creator with a website, this is Anthropic's own stated opt-out: a ClaudeBot disallow rule in robots.txt. Blocking only ClaudeBot keeps the site eligible for Claude's search-grounded answers and user-initiated fetches, which is the middle ground most site owners want. Note the limitation Anthropic states itself: the opt-out covers "future materials" — it does not describe any mechanism for removing content already collected.

What is not yet known

The documentation does not say how long it takes for an opt-out to propagate into training datasets, nor does it offer a way to verify exclusion. As with all robots.txt mechanisms, compliance is voluntary on the company's part.

Sources