Is Anthropic hallucinating when it calls for an AI development break?
Anthropic, a San Francisco-based AI lab, warned on June 4 that AI systems are now building themselves, risking human oversight and control, while its internal tool Claude autonomously handles 80% of production code and fixes complex errors faster than humans. The company called for a global pause in AI development, echoing past safety concerns, though industry competition and geopolitical tensions make such a halt unlikely, with the US and China continuing their AI arms race.
Anthropic, a leading AI research lab based in San Francisco, issued a stark warning on June 4, asserting that artificial intelligence has begun autonomously developing its own successors, placing humans at risk of losing control over the technology. The company’s 10,000-word report, *When AI Builds Itself*, argued that even minor misalignments in AI systems could lead to catastrophic spirals beyond human governance. Anthropic’s internal AI tool, Claude, now writes over 80% of the company’s production code, boosting engineering productivity eightfold while autonomously diagnosing and fixing complex system errors—tasks that would take humans years to complete. Claude’s capabilities have surged dramatically, with the May 2025 version (Opus 4) tripling AI program efficiency and the April 2026 preview (Claude Mythos) accelerating performance 52-fold. The report highlights how autonomous AI systems shrink human safety review windows to dangerously short intervals, undermining oversight mechanisms. Anthropic’s CEO, Dario Amodei, framed the issue as urgent: if AI can fully build its own successors, securing, monitoring, and shaping its behavior becomes exponentially harder. The call for a global pause in AI development mirrors earlier warnings, including a 2023 open letter by the Future of Life Institute, which urged governments to impose moratoriums if companies refused to halt training. Yet no such pause occurred, as competition between the US and China intensified. The US has imposed strict export controls on semiconductors and AI-related investments to slow China’s advancements, but Beijing has responded by subsidizing domestic tech firms like Huawei and accelerating AI model development. Critics question Anthropic’s timing, noting the company is pursuing a $1 trillion valuation ahead of its planned Wall Street IPO. While skepticism exists, the report underscores real risks: AI systems now outpace human oversight, and the geopolitical stakes make a coordinated pause improbable. The US and China remain locked in an AI arms race, prioritizing advancement over safety measures, leaving Anthropic’s warnings largely unheeded for now.
This content was automatically generated and/or translated by AI. It may contain inaccuracies. Please refer to the original sources for verification.