Study finds AI can match clinicians in interview assessment
This image was generated by AI and may not depict real events.
Researchers in Japan found that AI can match clinicians in assessing medical interview skills, potentially making medical training fairer, faster, and more scalable. The study used AI models to evaluate transcripts of clinical interviews, showing strong agreement with human evaluations and greater consistency.
A study published in JMIR Medical Education found that artificial intelligence (AI) can effectively assess medical interview skills, a crucial aspect of physician training. Researchers from Juntendo University Faculty of Medicine in Japan compared AI-based assessment (ABA) with traditional human-based assessment (HBA) using transcripts of clinical interviews. Seven participants, including medical students and physicians, conducted interviews with an AI-simulated patient, which were then evaluated using the Master Interview Rating Scale. AI models, specifically GPT-o1 Pro and GPT-5 Pro, were used for ABA, while five experienced clinical instructors performed HBA. The results showed strong agreement between ABA and HBA, with AI demonstrating greater consistency and reducing evaluation time by more than half. The researchers suggest a practical 'AI-first, faculty-verified' model, where AI handles initial evaluations, and educators focus on coaching and high-stakes decisions. While AI showed promise, the researchers emphasize the need for human oversight, as transcript-based evaluation cannot capture nonverbal cues and cultural nuances. The study highlights the potential of AI to improve medical education by providing rapid and consistent feedback, making repeated practice more accessible, particularly in settings with limited faculty resources.
This content was automatically generated and/or translated by AI. It may contain inaccuracies. Please refer to the original sources for verification.