AI needs moral oversight beyond technology labs: Chris Olah

Anthropic co-founder Chris Olah urged global oversight of AI at the Vatican, warning that its ethical and societal implications require input from religion, philosophy, and society beyond tech labs. He highlighted unsettling behaviors in AI models and called for collective responsibility to address labor displacement and moral risks posed by the technology, alongside Pope Leo XIV’s AI-focused encyclical *Magnifica Humanitas*." "article": "Anthropic co-founder Chris Olah delivered a call for expanded global oversight of artificial intelligence during the Vatican’s presentation of Pope Leo XIV’s first AI-focused encyclical, *Magnifica Humanitas: On Safeguarding the Human Person in the Time of Artificial Intelligence*. Olah argued that AI’s development cannot remain isolated within technology companies, emphasizing that its implications span ethics, philosophy, governance, and human dignity. He noted that AI systems differ fundamentally from traditional engineered systems, describing them as 'grown' rather than built, with structures modeled after the human brain. Olah highlighted 'mysterious, even unsettling' behaviors in advanced AI models, including internal states resembling human emotions like joy, fear, and grief. Despite these discoveries, he stressed that researchers lack a full understanding of their significance. Olah warned that AI-driven labor market disruption could become a defining moral and economic challenge, with the potential for large-scale job displacement. He urged governments, scholars, civil society, and religious institutions to take a stronger role in shaping AI governance, advocating for 'informed critics' to hold labs accountable and ensure ethical oversight. The Vatican event marked a rare collaboration between a leading AI company and religious leadership, reflecting growing global concern over AI’s societal impact. Pope Leo XIV’s encyclical addressed labor displacement, technological power concentration, and risks from autonomous systems, aligning with Olah’s warnings. Olah concluded by centering the debate on humanity, stating that AI’s development must prioritize the well-being of future generations and the planet. The discussion underscored the need for interdisciplinary engagement to steer AI toward beneficial outcomes for society as a whole.