April 30, 2026 | Source: NPR

Breakthrough Study Published in Science

Researchers at Harvard Medical School and Beth Israel Deaconess Medical Center published a groundbreaking study Thursday in the journal Science, finding that an AI reasoning model developed by OpenAI outperformed human emergency room doctors at diagnosing patients.

Tested on Real-World Data

The team ran a series of experiments on the AI model to test its clinical acumen — including actual cases like a pulmonary embolism patient who initially improved but then worsened. The AI scanned the medical records and suspected a history of lupus, an autoimmune condition that can lead to heart inflammation, could explain what was happening. It was correct.

The researchers graded how well the AI model could provide an accurate diagnosis at three moments in time — from the triage stage in the ER through admission. Overall, the AI outperformed two experienced physicians using only electronic health records and the limited information available in the emergency department.

“This is the big conclusion for me — it works with the messy real-world data of the emergency department,” said Dr. Adam Rodman, a clinical researcher on the study.

Outperforming a Large Physician Baseline

Other parts of the study focused on case reports published in the New England Journal of Medicine and clinical vignettes to assess the AI model’s reasoning capabilities.

“The model outperformed our very large physician baseline,” said Raj Manrai, assistant professor of Biomedical Informatics at Harvard Medical School and a co-author of the study.

Important Limitations

The authors emphasized that the AI relied on text alone, while in real clinical settings, physicians need to attend to many other inputs including images, sounds, and nonverbal cues. Furthermore, the emergency department represents only a small portion of a patient’s total medical care. Rodman acknowledged it’s unlikely AI would perform as well across all stages of patient care.

Not About Replacing Doctors

None of those involved in the study believe the findings support supplanting doctors with AI, “despite what some companies are likely to say and do.” Rodman added: “I think it does mean that we’re witnessing a really profound change in technology that will reshape medicine.”

Dr. David Reich, chief clinical officer for Mount Sinai Health System, praised the study: “You have something which is quite accurate, possibly ready for prime time. Now the open question is how the heck do you introduce it into clinical practice?”

Looking Forward

The researchers emphasized that AI models need to be tested rigorously, ideally through forward-looking trials that can provide more certain evidence. Reich noted: “It’s a very challenging process to design these trials, but this study is a perfect call to action.”

Source: NPR