Anthropic calls for global AI slowdown after $965B valuation. Critics claim it’s just to hobble competition.
Anthropic, now the most valuable AI company with a $965 billion valuation, called for a global pause in AI development to address risks like recursive self-improvement, while critics accuse the firm of using safety concerns as a competitive tactic. The company’s proposal includes potential verification mechanisms for a slowdown, though rivals like OpenAI’s Sam Altman dismiss such warnings as fear-based marketing.
Anthropic, the AI company valued at $965 billion following its $65 billion Series H funding round last month, has urged the industry to slow the pace of AI development. In a blog post published Thursday, the company warned that AI systems could soon achieve 'recursive self-improvement,' meaning they could enhance their capabilities without human oversight, posing potential societal risks. The post, authored by Anthropic’s head of internal research Marina Favaro and head of policy Jack Clark, argued that a temporary pause in frontier AI development could allow societal structures and alignment research to keep pace with technological advancements. Anthropic proposed a global agreement to enforce such a slowdown, including verification mechanisms to ensure compliance. Critics, including OpenAI CEO Sam Altman, have dismissed similar warnings from Anthropic as fear-based marketing. Altman previously compared the company’s approach to selling bomb shelters after building a bomb. Some academics, like University of Pennsylvania’s Wharton School professor Ethan Mollick, have also questioned the sincerity behind Anthropic’s concerns, describing them as a mix of marketing and genuine beliefs about AI’s future. Anthropic’s call for caution comes amid its rise as a dominant force in AI, surpassing OpenAI in valuation. The company has repeatedly highlighted risks associated with its own models, including potential misuse in military applications, which led to tensions with the White House. CEO Dario Amodei has clashed with U.S. officials over these concerns, further intensifying debates about AI governance. The blog post acknowledged that self-improving AI is not yet a reality but suggested it could revolutionize industries while also enabling harmful applications, such as authoritarian surveillance or large-scale influence operations. Anthropic’s proposal reflects broader industry debates about balancing innovation with ethical and safety considerations in AI development.
This content was automatically generated and/or translated by AI. It may contain inaccuracies. Please refer to the original sources for verification.