Anthropic's Olah says AI must be guided from outside Big Tech

Anthropic co-founder Chris Olah addressed the Vatican on May 25, advocating for external oversight of artificial intelligence development to mitigate risks like widespread job displacement. Speaking at the launch of Pope Leo’s first encyclical on AI challenges, Olah warned of a 'real possibility' that AI could displace labor at an unprecedented scale, framing it as a 'moral imperative' to support affected workers. He noted that AI labs, including his own, operate under commercial and geopolitical pressures that may conflict with societal interests, emphasizing the need for outside scrutiny. Olah, a Canadian, became the sole Big Tech representative at the event, citing his focus on AI safety and engagement with religious communities. Anthropic, founded in 2021 by Olah and former OpenAI employees, has clashed with the U.S. government over military and surveillance applications of its AI tools, like Claude. The Vatican’s invitation to Olah stemmed from his career-long work on AI safety and dialogues with over 15 religions, though he acknowledged the selection was ultimately the Vatican’s decision. During his speech, Olah described AI’s rapid development as a 'scary moment,' urging collective efforts to steer the technology toward ethical outcomes. He identified three critical areas needing immediate attention: job displacement risks, global equity in AI benefits, and the interpretability of complex AI systems. Olah also questioned how to ensure AI advancements in wealthy nations are shared worldwide, framing it as a pressing ethical concern. The event marked an unusual collaboration between the tech sector and the Catholic Church, positioning the Vatican as a moral authority on AI’s implications. Olah praised the Church’s engagement with AI ethics, noting that concerns extend beyond technical challenges to broader societal impacts. He stressed that public anxiety, especially among young people, is justified by AI’s accelerating pace and potential risks, calling for proactive measures to prevent harmful outcomes.