CARing teaches LLMs to predict diagnoses: R@30 hits 46.52% on MIMIC-IV

The authors of arXiv preprint 2610.10641 propose CARing, a framework that pushes a language model to cover the full set of diagnoses that may be valid at the same time, rather than predict a single "most likely" diagnosis for a patient's next visit. On the MIMIC-III and MIMIC-IV datasets, the model beat every baseline trained on electronic health records (EHR) in weighted F1 and showed the best top-k recall at every reported threshold. In reasoning mode, R@30 reached 46.04% on MIMIC-III and 46.52% on MIMIC-IV.

Two problems the authors tackle at once

The first concerns reinforcement learning. The standard RL recipe for reasoning models rewards a trajectory for a correct final answer. That works for tasks with a single right result. But a patient at their next visit may have several diagnoses at once, and a "one diagnosis per trajectory" reward cannot tell a repeated hit on the same code from coverage of different ones. The policy quickly finds a couple of easy-to-guess diagnoses and circles around them. The rest stay out of view.

The second problem is tokenization. A typical LLM tokenizer splits an ICD code into several generic tokens with no clinical meaning. Producing a single diagnosis takes several decoding steps, and reasoning over a large vocabulary of diseases is awkward with this encoding.

How CARing works

The authors represent diagnoses as composite Semantic IDs (SIDs). First, disease semantics enriched with an ontology are encoded into compact identifiers through residual quantization. Then the SID tokens are "grounded" in natural language and in the longitudinal EHR context through multi-task alignment and training on reasoning data. The goal is to preserve the transferability of the LLM's reasoning ability without losing it on a specialized vocabulary.

For unordered multi-label prediction, the authors add two mechanisms: a coverage reward in RL and multi-positive supervision. The first rewards trajectories that expand the set of covered diagnoses. The second accounts for several correct labels at once in supervised training.

Two modes are available at inference:

• direct constrained decoding, a fast path without long reasoning;
• multi-chain reasoning with rank fusion, which runs several reasoning chains and merges their results into a single ranking of diagnoses.

What the numbers show

The authors report the following:

• Benchmarks: MIMIC-III and MIMIC-IV.
• Weighted F1: higher than all baselines trained on EHR data.
• Top-k recall: best result at every reported cutoff threshold.
• R@30 in reasoning mode: 46.04% on MIMIC-III and 46.52% on MIMIC-IV.

The preprint's abstract does not say exactly how far CARing leads its competitors in weighted F1, so the size of the gap has to be checked in the full text and the logs. The comparison is limited to models trained on EHR data. That is an important caveat: it is not a comparison against general-purpose frontier models without fine-tuning.

Where this is useful and where it is still too early

The coverage reward idea reaches beyond medicine. Any task where one input has several correct outputs suffers from the same collapsing reward: recommender systems, entity extraction, multi-label ticket classification, and an agent's tool-call prediction. If you train a reasoning model with RL yourself, check whether your reward function rewards repeating the same success.

Semantic IDs via residual quantization are the second transferable technique. They shorten the output length per entity and give the model a vocabulary with semantic structure. For teams with large closed reference sets (codes, SKUs, catalogs), this is a way to avoid fighting the tokenizer.

This work is far from the clinic. MIMIC is research data from a single healthcare system, and an R@30 of about 46% means that more than half of the relevant diagnoses do not make the top 30. As a decision-support tool and a base for further experiments, CARing is interesting. As a ready-made clinical product, it is not. The authors published the code and logs on GitHub (zmlxzyh/CARing-Codes-Logs), so the results can be checked independently.