Clinical reasoning study finds gaps beneath high average AI scores

A new study from IO Lab evaluated 21 large language models and found that strong average scores could coexist with weaknesses in specific clinical reasoning tasks. The study, published in JAMA Network Open, identified differential diagnosis as a particularly difficult area. This is the stage when a clinician considers several possible explanations for a patient’s presentation and decides what information to gather next.

Arya Rao and colleagues tested the models on 29 clinical cases from the MSD Manual. They collected 16,254 responses across tasks involving differential diagnosis, testing, final diagnosis, management and other reasoning. Cases unfolded in stages, and the models retained information supplied earlier. Web search and external retrieval were turned off.

Overall accuracy ranged from 81% to 90%, but results differed across task types. Every model had a failure rate above 80% for differential diagnosis questions. Failure rates for final diagnosis were below 40%. The researchers also used a composite measure designed to reflect balanced performance across the reasoning tasks.

The meaning of failure is essential to interpreting those figures. A question counted as failed when the response was not fully correct. For questions with several options, full credit required including all correct choices and excluding incorrect ones. The reported failure rates therefore describe performance under that scoring rule. They cannot be read as the percentage of real patients a chatbot would misdiagnose.

The findings have implications for how medical AI is assessed. A single average can hide weaknesses in tasks that influence later decisions. A plausible final diagnosis may also depend on information already supplied by a case author. Evaluations that examine the entire sequence can help identify where a system’s reasoning becomes incomplete and where additional investigation is needed.

Several features limit the study’s application to clinical practice. The cases were publicly available, so exposure during model training was possible. The tests evaluated baseline systems without external tools, and they did not measure clinicians working with AI or outcomes for patients. The findings should also be tied to the model versions and testing conditions used.

Further evaluation in clinical settings would need to examine unfamiliar cases, uncertainty and the consequences of errors. For hospitals considering decision support, the study provides a reason to ask for results broken down by clinical task. Those details can make a high overall benchmark score more informative.