Anthropic pitches robots.txt opt-out at Australia's AI inquiry; ABC and SBS push back

What happened

Anthropic has asked Australia's parliament for a "narrow form of conditional approval" to train AI models on copyrighted material, so long as rights holders can block its crawlers via robots.txt, in a submission to the Joint Select Committee on Artificial Intelligence.

Australia's two public broadcasters pushed back in their own submissions to the same inquiry. SBS told the committee that "robots.txt mechanisms are routinely ignored or bypassed" and that "the onus should be on AI companies to avoid wrongdoing" rather than on publishers to police the open web; it said large language models including ChatGPT and Claude use news content "without permission, attribution or payment." The ABC framed the imbalance as competition policy, calling the current treatment "anti-competitive" and warning it will "accelerate the cannibalisation of the Australian news industry"; it wants AI firms held to the same copyright, defamation, and privacy rules as media outlets. Both broadcasters want AI companies brought under Australia's news bargaining incentive, which currently covers Facebook, Google, and TikTok but not AI firms.

Why it matters to creators

Anthropic is asking a government to bless opt-out-by-robots.txt as the legal mechanism for consent — the same mechanism Australian publishers say is routinely ignored. Whatever Australia decides will shape the template for creator opt-out regimes worldwide: if a government accepts robots.txt as sufficient consent infrastructure, creators get almost nothing; if it rejects it, AI companies face real obligations to obtain permission or pay.

What is not yet known

Australia's government has already rejected a blanket text-and-data-mining exception; the committee is due to report in November. Anthropic and OpenAI executives are due to appear in Sydney next week, though CEOs Sam Altman and Dario Amodei are skipping a parallel Senate inquiry.