Anthropic CEO: Government should have power to block dangerous AI deployments

Anthropic CEO Dario Amodei published an essay Wednesday advocating for government power to block AI deployments that fail safety standards, framing regulation after the model of aviation oversight. He proposed mandatory third-party audits for frontier AI systems, including tests for cybersecurity risks, biological weaponization potential, loss of control, and automated research risks. If audits reveal threats, governments should halt or reverse deployments, Amodei argued, going beyond President Trump’s recent executive order, which established voluntary pre-release testing for AI models up to 30 days before public release. Amodei’s stance contrasts with industry peers wary of government overreach and aligns with Anthropic’s prior clashes with the Trump administration. Earlier this year, Defense Secretary Pete Hegseth labeled Anthropic a supply chain risk and barred its AI, Claude, from Pentagon use, prompting a lawsuit from Anthropic. The company demanded restrictions on autonomous lethal weapons and mass surveillance, while the Pentagon insisted on unrestricted access for ‘lawful uses.’ The dispute intensified after Anthropic released Mythos, its advanced cybersecurity AI model, raising concerns in Washington about AI safety risks. Amodei warned that future AI systems could pose existential threats akin to nuclear materials, necessitating stricter policies. His proposal suggests government agencies could conduct audits, a departure from industry norms favoring private oversight. Trump’s executive order, signed last month, fell short of mandatory testing, leaving critics skeptical about preventing overregulation. Amodei’s essay positions Anthropic as a proponent of preemptive safeguards, even as its own products face scrutiny. The company’s legal battle with the Pentagon and its recent AI advancements underscore the tension between innovation and regulatory caution in the AI sector.