AI Lawyers Are Already Better Than Law Professors at Reasoning—Say Law Professors

A study led by Stanford University found that artificial intelligence outperformed human law professors in generating contract law answers. Sixteen professors from 14 U.S. law schools—including Stanford, Yale, and New York University—created 40 contract law questions covering legal doctrine, case law, and policy issues. Researchers tested how often professors preferred AI-generated answers over their own, using blinded comparisons. In 2,918 evaluations, Google’s Gemini 2.5 Pro and NotebookLM won 75.92% and 74.75% of matchups, respectively, against human instructors. The study also found AI answers were flagged as harmful less often—3.41% for Gemini and 3.64% for NotebookLM—compared to 12.06% for human responses. Additional models like Anthropic’s Claude Opus 4.7 and OpenAI’s ChatGPT 5.4 also outperformed professors on average. Researchers analyzed whether AI success stemmed from writing style or substantive content, testing features like answer length, clarity, and legal reasoning. They concluded that AI alignment with disciplinary standards—not just superficial qualities—explained the preference. However, the study noted that AI answers might not perfectly match individual teaching preferences, only that they were generally viewed as acceptable. The findings suggest large language models could serve as effective educational tools in law, though further research is needed to assess their adaptability to specific instructional approaches. The study highlights AI’s potential in legal education while acknowledging limitations in personalized tailoring.