Your lawyer might be filing 'hallucinating' AI information on your case

The Florida Supreme Court has tightened rules to prevent attorneys from filing court documents containing AI-generated false legal citations, known as 'hallucinations.' In a recent opinion, the court warned that generative AI tools, while useful for drafting, frequently produce fabricated authorities—up to 33% of the time, according to a study cited by a Florida appellate court. The new rules require lawyers to certify that all cited legal precedents exist and are accurate, allowing courts to impose sanctions for violations. Florida joins other states like Illinois, Ohio, and Pennsylvania in addressing AI misuse in legal filings. The state’s high court acted after lower courts reported growing concerns about attorneys unknowingly or intentionally relying on AI-generated fictions. John Morgan, founder of Morgan & Morgan, admitted his firm discovered AI risks early when a lawyer included fabricated cases in a federal filing, prompting immediate internal safeguards. Florida’s Attorney General, James Uthmeier, separately sued OpenAI, alleging the company prioritized profits over safety by collecting data from minors and misrepresenting ChatGPT’s capabilities. Meanwhile, President Trump signed an executive order calling for greater oversight of AI models, reflecting broader regulatory scrutiny. Legal experts, including Orlando attorney Brent Hartman, emphasized that judges depend on accurate filings to make rulings, as they cannot verify every cited precedent. Judge Jonathan Lott of Florida’s Fourth District Court of Appeal noted that AI tools predict plausible-sounding responses but lack true understanding, often producing errors. The court’s decision aims to establish clearer standards amid rising AI adoption in legal research and drafting. Other states, such as New York, New Jersey, and Connecticut, have formed task forces to study AI’s impact on court proceedings. The Florida Supreme Court’s ruling underscores the need for uniform guidelines as generative AI becomes more prevalent in legal practice.