Multimodal Alzheimer's AI model reaches 0.935 AUROC on 2649 patients

Researchers have posted a multimodal, multitask system for diagnosing Alzheimer's disease and predicting its progression to arXiv (cs.LG). Training and internal validation used 2649 patients and 17 317 visits from three datasets: ADNI, OASIS-2 and MIRIAD. Diagnostic AUROC was 0.935 ± 0.002 (mean over three seeds), and the hybrid version reached 0.951. On the external OASIS-3 cohort the results are lower: 0.763 for the neural network and 0.767 for the hybrid. That gap is the main story of the paper.

Architecture and leakage protection

Alzheimer's data is awkward to work with: visits are irregular, measurements are heterogeneous, and some patients are missing some modalities. The authors built a pipeline from several blocks. An adapted SFCN encoder processes the MRI scans. Clinical data passes through four causal transformers. After that come a shared fusion layer and separate dynamic ODE-GRU modules for each task, which can handle uneven intervals between visits.

The last two blocks of the MRI encoder are adapted through fine-tuning and LoRA. Task-DRO balances the losses across tasks. Group-CVaR deliberately lifts performance across cohorts and across strata defined by comorbidities, that is, co-occurring diseases.

The paper's title puts particular emphasis on leakage control. To achieve it, the authors use input constraints at the level of individual branches, a strictly subject-level split of the data, and empirical causality checks. For longitudinal data this is critical: if visits from the same patient end up in both train and test, the metrics come out inflated.

Numbers and external validation

Main results from internal validation and OASIS-3:

• Diagnosis: AUROC 0.935 ± 0.002, hybrid 0.951.
• Stage-1 progression: 0.884 ± 0.003, hybrid 0.909.
• Stage-1 progression from the first visit: 0.870 ± 0.005, hybrid 0.896.
• MMSE prediction error at the next visit: MAE of 1.61 points.
• Worst stratum for diagnosis: AUROC 0.827 ± 0.008.
• OASIS-3, diagnosis: 0.763 (network) and 0.767 (hybrid).
• OASIS-3, progression at the next visit (hybrid): 0.764.
• OASIS-3, diagnostic calibration error: reduced from 0.197 to 0.052.
• OASIS-3, MMSE MAE: 0.86.

The drop in AUROC from 0.935 to roughly 0.76 is a classic domain shift between cohorts. Calibration on external data, however, improved almost fourfold, which matters at least as much for clinical use as ranking quality does. The lower MMSE MAE on OASIS-3 (0.86 versus 1.61) may depend on the composition of the sample. The abstract does not name a cause, so this is only a hypothesis.

The ablations look surprising. The authors disabled six components one at a time on a single seed (42), and the change in combined AUROC for diagnosis and progression ranged from −0.004 to +0.004. No single architectural element noticeably moves the metric. Removing the clinical inputs, however, lowers diagnostic AUROC by 0.272. The main signal apparently lies in the clinical data. Explanations on the ADNI sample also show that different tasks rely on different inputs.

What to take away

This is a preprint without peer review, and it is not a clinical product. Its practical value lies elsewhere: the paper sets out a careful evaluation protocol. A subject-level split, independent per-branch input checks, worst-stratum reporting, and external validation with calibration are all worth adopting for teams building medical models on longitudinal data.

For those choosing an architecture, the ablation results suggest that making the model more complex will deliver less than improving the quality of the clinical features. To judge real-world applicability, look at results on independent cohorts like OASIS-3, not at the internal AUROC of 0.935. The numbers there are noticeably more modest, but also more honest.