The Hacker Sent by Anthropic to Calm the Government’s Nerves About AI Safety

Anthropic researcher Nicholas Carlini, a renowned cybersecurity skeptic, recently shifted his stance after discovering AI models could rapidly identify and exploit vulnerabilities in widely used software. In March, he presented findings at a San Francisco conference, showing how Anthropic’s Mythos AI uncovered critical bugs in web-publishing software Ghost and the Linux operating system—systems previously considered secure. Carlini, who had never found such flaws before, warned that AI was surpassing human vulnerability researchers, signaling a potential collapse of traditional cybersecurity defenses. The revelation led to a delay in Mythos’ release and sparked a broader panic among cybersecurity experts, dubbed 'Bugmageddon.' Anthropic later released Mythos 5 and a restricted version, Fable 5, but the White House swiftly banned foreign entities from using either model. The U.S. government’s move forced Anthropic to halt access globally, escalating tensions between the company and the Trump administration. Carlini, now a vocal advocate for cautious AI deployment, was dispatched to Washington to address government concerns. He argued that controlled release of Fable 5—despite inherent risks—was preferable to secrecy, aiming to ease fears about AI-driven cyber threats. His role reflects the broader struggle to balance innovation with security in an era of rapidly advancing AI capabilities. The incident also reignited conflicts between Anthropic CEO Dario Amodei and Defense Secretary Pete Hegseth, who had previously clashed over military use restrictions and AI export policies. Earlier this year, Amodei pushed the Pentagon to stop using Anthropic models, leading to lawsuits and further straining relations with the administration. With AI development accelerating, Carlini’s transformation from skeptic to advocate underscores the urgent need for collaboration between tech firms and policymakers. The debate over Mythos and Fable 5 highlights the challenges of regulating AI while preventing misuse, as both sides grapple with the implications of unchecked technological progress.