We need to stop AI developing without humans, says Anthropic co-founder

Anthropic co-founder Jack Clark has urged governments to implement policies that allow society to slow or halt AI development, comparing the need for regulation to historical responses to the oil industry. He warned that AI systems, like Anthropic’s chatbot Claude—which already uses 80% self-written code—could reach full autonomy within two years, raising risks of economic disruption and misuse. Clark emphasized the lack of a 'brake pedal' in current AI progress, stating that without intervention, companies may continue advancing without adequate oversight. The recent US executive order on AI, issued by President Donald Trump, was criticized by Clark for its voluntary approach, as it does not require mandatory safety testing or government oversight of AI firms. Major companies, including Anthropic, OpenAI, and Google, have not committed to pausing research, despite concerns over job losses and autonomous AI agents handling routine tasks. Clark linked these fears to recent mass layoffs in tech, where AI tools have replaced human workers in software engineering and other roles. Anthropic, founded five years ago by CEO Dario Amodei and Clark, has positioned itself as a leader in advocating for AI risk mitigation, even clashing with the US Department of Defense over potential military applications. Clark expressed alarm for future generations, stating that society must address AI’s implications before irreversible consequences arise. He acknowledged potential benefits but stressed the need for regulation to prevent misuse, such as mass surveillance or autonomous warfare. The company is preparing for a high-profile public stock market debut, with a valuation nearing $1 trillion, making it one of the most anticipated IPOs in history. Clark clarified that Anthropic’s warnings are not aimed at boosting its reputation but at raising awareness about the technology’s rapid, unchecked advancement. The call for regulation follows broader debates on AI safety, with experts warning of economic and ethical risks if development continues without safeguards.