Anthropic CEO Amodei proposes government authority to block risky AI models
In brief
- Anthropic CEO Dario Amodei proposed regulatory blueprint granting government authority to block unsafe AI models before deployment
- Framework mandates third-party testing for frontier AI models with independent auditors stress-testing for dangerous capabilities
- Revenue-based civil penalties for noncompliance with graduated regulatory measures as AI capabilities advance
- Proposal includes Economic Policy Framework to support workers displaced by AI-related job losses
The Core Proposal
The framework proposes that independent auditors stress-test models for dangerous capabilities, similar to how pharmaceuticals undergo clinical trials or aircraft face safety certification before carrying passengers. If a model fails those tests, the government could block its deployment entirely.
The proposal targets AI models capable of enabling cybersecurity threats or aiding in the development of biological weapons. The framework also proposes significant civil penalties tied to a company's global annual revenue for noncompliance. A company generating tens of billions in annual revenue could face penalties in the hundreds of millions or more for deploying unsafe models.
Safety as Competitive Advantage
Amodei's proposal reflects Anthropic's own stance on AI safety. In February, Anthropic refused Pentagon demands to remove safety guardrails from the Claude family of models. Those guardrails are specifically designed to prevent the use of AI in autonomous weapons systems and domestic mass surveillance. The refusal sparked a legal dispute centered on supply-chain risks.
The regulatory framework could create a meaningful competitive moat for companies that have already invested heavily in safety infrastructure. Companies like Anthropic, which have built safety considerations into their development process from day one, would have a head start. Firms that have been treating safety as an afterthought will face steep adjustment costs if third-party audits become mandatory.
Graduated Approach and Worker Support
Amodei is advocating for a graduated approach, starting with softer regulatory measures that could intensify over time as AI capabilities advance. The proposal includes an Economic Policy Framework aimed at supporting workers displaced by AI-related job losses.
Anthropic's refusal to strip safety features suggests that some companies may be willing to walk away from government dollars rather than compromise on principles. The Advanced AI Framework signals a shift in how the industry might approach regulation—one where safety-first companies gain structural advantages and compliance becomes a baseline competitive requirement, not an optional cost.
