Harvard: AI beats doctors at emergency-room triage diagnosis

A team from Harvard Medical School and Beth Israel Deaconess Medical Center reported that OpenAI's o1 reasoning model was more accurate than physicians in a number of emergency diagnostic tasks. The most striking result came at initial triage in the emergency department, when data is scarce and a decision is needed fast.

How the comparison worked

The study was published on 30 April 2026 in the journal Science and is one of the largest attempts to compare AI not with exam-style tests but with real clinical work. The authors ran the model through six experiments, from difficult diagnostic cases and probabilistic reasoning to tasks on choosing next steps.

The key part of the work was 76 real cases from the emergency department of a Boston hospital. The model and the physicians received the same electronic health record entries and had to propose the most likely diagnoses and next steps.

Crucially, the data was barely cleaned up before the test. The researchers used the same noisy, incomplete text a doctor sees in the first minutes: vital signs, age, a nurse's brief description of complaints, and scattered notes from the medical history. Evaluation covered three stages: at triage, at first contact with a physician, and at the decision to admit the patient to a ward or intensive care. The evaluators did not know whether a human or the model had given a given answer.

Where AI came out ahead

The model's biggest advantage appeared exactly where the physician has the least information. At early triage, OpenAI o1 gave an exact or very close diagnosis in 67% of cases. Physicians on the same set of patients scored in the 50–55% range. As more data became available, the AI's accuracy rose to 82% and the humans' to 70–79%; here the gap was no longer statistically significant, but the trend held.

In case-management tasks, including choosing tests and antibiotics and discussing treatment goals, the model also performed markedly better.

• 67%: exact or close diagnosis by the AI at initial triage
• 50–55%: physicians' result at the same stage
• 82%: AI accuracy once additional data arrived
• 89%: the model's score on management-planning tasks, versus 34% for physicians

The authors also cited an illustrative example. In one case, a patient arrived with a pulmonary blood clot and a worsening condition. The physicians assumed standard anticoagulant therapy had failed. The model instead linked the picture to a history of lupus and suggested the source of the problem was lupus-related inflammation of the lungs. That version was later confirmed. The researchers separately note that the model worked especially confidently on rare and tangled cases.

Why this is not a replacement for doctors

These results do not mean the emergency department can be put on autopilot. The study tested mainly the text portion of clinical reasoning: reading the chart, building a differential diagnosis and proposing a next step. The AI did not examine the patient, did not see the expression of pain, did not listen to breathing, did not assess gait, and did not work with X-rays and ECGs the way a doctor does at the bedside. Outside experts have already pointed out that this is closer to a "blind second opinion" on text than to managing a patient in real time.

"We are seeing a truly profound technological shift that will change medicine," said study co-author

Arjun Manrai.

But the authors themselves stress the limitations. Even if the model more often guesses the main diagnosis, it may suggest unnecessary tests or interventions that harm the patient. There is also no clear accountability framework yet: who is responsible for an error if a doctor trusted the algorithm's suggestion?

The researchers therefore speak not of replacing doctors but of a new format of joint work, in which AI acts as a fast analyst and source of a second opinion while the final decision stays with a human.

What this means

For medicine, this is a signal that large language models are moving out of demo mode and approaching real clinical validation. The most likely near-term scenario is not an autonomous "AI doctor" with no humans, but systems that quietly scan electronic records, flag missed diagnostic possibilities and help prioritize faster in the emergency department.

The next step is now clear: not new benchmarks, but prospective clinical trials that look beyond answer accuracy to safety, cost and impact on treatment outcomes.