Research raises concerns about impact on justice systems of using AI

Irish researchers have uncovered significant vulnerabilities in AI systems used for legal decision-making, revealing that these models can be heavily influenced by the quality of legal arguments presented. A study by Oisin Suttle of Maynooth University’s School of Law and Criminology and David Lillis of UCD’s School of Computer Science found that AI judges in simulated legal contests were measurably persuadable, with stronger advocate models winning between 58% and 71% of cases on average. In extreme cases, the stronger advocate won over 90% of the time, demonstrating how easily outcomes could be swayed by persuasive argumentation. The research, titled *Persuadability and LLMs as Legal Decision Tools*, is the first systematic study of AI persuadability in legal settings. It tested frontier AI models from Anthropic, Google, OpenAI, Mistral, and DeepSeek using real appellate court cases from Ireland, England and Wales, and the United States. The experiment pitted AI models against each other, with one acting as an advocate for each side and a third as the judge, measuring how often the stronger argument prevailed. The findings raise concerns about fairness in justice systems, as parties with access to sophisticated legal representation or advanced AI advocates could gain a systematic advantage. Even larger, more capable models showed significant sensitivity to argument quality, though they were slightly less persuadable than smaller models. However, the researchers warned that smaller models’ reduced persuadability may stem from an inability to evaluate arguments rather than independent judgment. The study highlights a critical gap in AI governance, as courts and tribunals worldwide are increasingly piloting AI for tasks like case triage and judgment drafting. The researchers argue that persuadability must be measured, disclosed, and addressed before AI is widely adopted in legal decision-making. They caution against unchecked optimism about AI capabilities, emphasizing the need for rigorous scientific assessment in sensitive areas like justice. While not advocating against AI use in legal settings, the study warns that unchecked persuadability could undermine fairness and reliability. The findings coincide with Ireland’s judiciary preparing new guidance on AI use, underscoring the urgency of addressing these risks before AI becomes entrenched in legal processes.