AegisDx: AI framework for safer diagnosis outscores GPT-5 in physician ratings

A research team posted a description of AegisDx on arXiv in July 2026. It is a framework for safe differential diagnosis built on large language models. In a blinded trial on 43 real patient records from the emergency department of the Yale New Haven Health System, physicians gave AegisDx an average safety score of 4.55 out of 5, against 4.31 for GPT-5. The difference is statistically significant (adjusted p = 2.1×10⁻⁴).

Why a plain LLM is not enough for safe diagnosis

Diagnostic errors remain one of the main threats to patient safety. Current LLM systems usually treat diagnosis as a "single query" task: the doctor describes symptoms and the model offers a guess. That approach has no safeguard against missing life-threatening conditions, and it does not verify its own chain of reasoning.

AegisDx works on a fundamentally different principle. The authors describe their approach as hypothetico-deductive clinical reasoning: the system first generates a broad list of hypotheses, then narrows it step by step, checking each step against the medical evidence base. The key elements of the architecture:

• Role contracts: each LLM component is responsible strictly for its own task
• Structured intermediate outputs: each reasoning step is recorded explicitly
• Evidence-retrieval interfaces: hypotheses are checked against the medical literature in real time
• Verification gates: the final diagnosis is checked before it is returned
• Mandatory must-not-miss screening: heart attack, pulmonary embolism (PE), aortic dissection and meningitis. The system must check them explicitly, so they cannot be skipped by accident

What the three levels of testing showed

The authors compared AegisDx with GPT-oss-120B used directly, without additional constraints. For each group of cases, the main metric was Top-3 accuracy: whether the correct diagnosis appears among the three best guesses, which matches real clinical practice.

• JAMA cases: Top-3 accuracy of 59.9% versus 52.1%
• NEJM cases: Top-3 accuracy of 62.7% versus 51.4%
• Annals of Emergency Medicine: 85.7% versus 68.6%
• Critical must-not-miss diagnoses: AegisDx included at least one such condition in its top 3 in 78.0% of cases, versus 52.0% for the standalone LLM

The strictest test was the blinded evaluation by clinician physicians: 43 real emergency department records were compared blind against GPT-5's conclusions. Participants rated AegisDx higher on the composite safety score, 4.55 versus 4.31 on a five-point scale. In qualitative feedback, they noted better detection of life-threatening diagnoses and more transparent reasoning.

What this means

The gain from a structured framework turned out to be comparable to that of scaling up the model directly, and the authors consider this the main takeaway. Engineering diagnostic AI systems as safe reasoning pipelines may be more productive than chasing raw predictive accuracy, especially in emergency medicine, where a diagnostic error can cost a life. AegisDx is positioned not as a replacement for the physician but as a transparent decision-support tool at the bedside.

Frequently asked questions

Which language model does AegisDx use?

In the experiments, the authors used GPT-oss-120B as the base model for all components. The framework's architecture is not tied to a specific LLM and was designed to be model-agnostic.

What clinical material was used in the trials?

The literature cases come from NEJM, JAMA and Annals of Emergency Medicine. The blinded trial with physicians used 43 real patient records from the emergency department of the Yale New Haven Health System.