Study Overview

A new study led by physicians and computer scientists at Harvard Medical School and Beth Israel Deaconess Medical Center was published this week in the journal Science. The research team conducted a series of experiments to evaluate how OpenAI’s AI models compared to human physicians across various medical scenarios.

Experimental Design

The study focused on 76 patients who visited the Beth Israel emergency room, comparing diagnoses from two internal medicine attending physicians to those generated by OpenAI’s o1 and 4o models. The diagnoses were independently assessed by two additional attending physicians who were blinded to whether each diagnosis came from a human or an AI.

The researchers emphasized that they did not “pre-process the data at all” — the AI models received the exact same information available in the electronic medical records at the time of each diagnosis.

Key Findings

The o1 model performed particularly well at the initial ER triage stage — the point at which patient information is most limited and decisions are most urgent:

• The o1 model provided the “exact or very close diagnosis” in 67% of triage cases
• The first physician achieved 55% accuracy
• The second physician achieved 50% accuracy

“We tested the AI model against virtually every benchmark, and it eclipsed both prior models and our physician baselines,” said Arjun Manrai, who heads an AI lab at Harvard Medical School and is one of the study’s lead authors.

Important Limitations

The researchers explicitly stated that the study does not claim AI is ready to make real life-or-death decisions in the emergency room. Instead, they called for “an urgent need for prospective trials to evaluate these technologies in real-world patient care settings.”

The study also noted that it only tested how models performed with text-based information, and that “existing studies suggest that current foundation models are more limited in reasoning over non-text inputs.”

Ethical and Accountability Concerns

Adam Rodman, a Beth Israel physician and co-lead author, warned that there is “no formal framework right now for accountability” around AI diagnoses, and that patients still “want humans to guide them through life or death decisions and through challenging treatment decisions.”

Kristen Panthagani, an emergency physician commenting on the study, called it “an interesting AI study that has led to some very overhyped headlines,” noting that the comparison was against internal medicine physicians rather than ER specialists. “If we’re going to compare AI tools to physicians’ clinical ability, we should start by comparing to physicians who actually practice that specialty,” she said.

She added: “As an ER doctor seeing a patient for the first time, my primary goal is not to guess your ultimate diagnosis. My primary goal is to determine if you have a condition that could kill you.”

Looking Ahead

This study marks a significant milestone in AI’s potential for medical diagnostics. However, substantial hurdles remain before clinical deployment, including regulatory approval, liability frameworks, and the ability of models to process non-text medical data such as imaging and lab results.

Sources: TechCrunch, Science

April 30, 2026 | Source: NPR

Breakthrough Study Published in Science

Researchers at Harvard Medical School and Beth Israel Deaconess Medical Center published a groundbreaking study Thursday in the journal Science, finding that an AI reasoning model developed by OpenAI outperformed human emergency room doctors at diagnosing patients.

Tested on Real-World Data

The team ran a series of experiments on the AI model to test its clinical acumen — including actual cases like a pulmonary embolism patient who initially improved but then worsened. The AI scanned the medical records and suspected a history of lupus, an autoimmune condition that can lead to heart inflammation, could explain what was happening. It was correct.

The researchers graded how well the AI model could provide an accurate diagnosis at three moments in time — from the triage stage in the ER through admission. Overall, the AI outperformed two experienced physicians using only electronic health records and the limited information available in the emergency department.

“This is the big conclusion for me — it works with the messy real-world data of the emergency department,” said Dr. Adam Rodman, a clinical researcher on the study.

Outperforming a Large Physician Baseline

Other parts of the study focused on case reports published in the New England Journal of Medicine and clinical vignettes to assess the AI model’s reasoning capabilities.

“The model outperformed our very large physician baseline,” said Raj Manrai, assistant professor of Biomedical Informatics at Harvard Medical School and a co-author of the study.

Important Limitations

The authors emphasized that the AI relied on text alone, while in real clinical settings, physicians need to attend to many other inputs including images, sounds, and nonverbal cues. Furthermore, the emergency department represents only a small portion of a patient’s total medical care. Rodman acknowledged it’s unlikely AI would perform as well across all stages of patient care.

Not About Replacing Doctors

None of those involved in the study believe the findings support supplanting doctors with AI, “despite what some companies are likely to say and do.” Rodman added: “I think it does mean that we’re witnessing a really profound change in technology that will reshape medicine.”

Dr. David Reich, chief clinical officer for Mount Sinai Health System, praised the study: “You have something which is quite accurate, possibly ready for prime time. Now the open question is how the heck do you introduce it into clinical practice?”

Looking Forward

The researchers emphasized that AI models need to be tested rigorously, ideally through forward-looking trials that can provide more certain evidence. Reich noted: “It’s a very challenging process to design these trials, but this study is a perfect call to action.”

Source: NPR

On April 29, 2026, Mayo Clinic released the validation results of a landmark study demonstrating that its AI model can detect “invisible” tissue changes of pancreatic cancer in routine CT scans, identifying disease indicators years before patients develop any symptoms. This breakthrough could fundamentally transform the early screening and diagnosis of pancreatic cancer.

The Technology: AI Outperforming Radiologists

Mayo Clinic’s research team used deep learning algorithms to train an AI model capable of recognizing subtle tissue changes in CT scans that are nearly imperceptible to the human eye. Bloomberg reported that the AI model “can find pancreatic cancer before anyone feels sick,” with detection accuracy exceeding that of experienced radiologists.

Medical Xpress noted that the model can detect tissue changes at “stage 0” of pancreatic cancer — ultra-early lesions that traditional medical imaging technology is virtually unable to identify. This means patients could be diagnosed at the stage when treatment is most effective, potentially increasing survival rates dramatically.

Validation Study: The 3-Year Detection Milestone

The validation study, published through the Mayo Clinic News Network, showed that the AI model successfully identified early lesion signals up to three years before confirmed diagnosis in retrospective analysis. This finding has been described as a “landmark validation,” providing robust data support for the clinical application of AI-assisted cancer screening.

Pancreatic cancer is known as the “king of cancers” due to its extremely high mortality rate — most patients are diagnosed at advanced stages when symptoms appear, with a five-year survival rate below 10%. If AI models can detect early lesions in routine CT scans during health checkups, survival rates could potentially multiply.

Clinical Prospects and Challenges

Health Tech World’s analysis pointed out that while the AI model demonstrates encouraging detection capabilities, several challenges must be overcome before widespread clinical deployment. These include validation of generalization across different medical institutions and CT equipment, large-scale prospective clinical trials, and regulatory approval processes.

Additionally, AI-assisted diagnosis faces practical challenges including false positive rate control, medical liability definition, and patient privacy protection. Mayo Clinic stated that the team is collaborating with multiple medical institutions on multi-center validation studies, aiming to obtain regulatory approval within the next two years.

Broader AI Healthcare Trends

This study represents the latest breakthrough in AI’s ongoing advancement in medical imaging. From lung nodule detection to breast cancer screening, AI models are demonstrating capabilities that surpass traditional methods across multiple disease areas. Mayo Clinic’s pancreatic cancer AI detection model has the potential to become another landmark achievement in the transition of AI healthcare from the laboratory to clinical practice.

Source: Mayo Clinic, Bloomberg, Medical Xpress