Doctors Don't Ask LLMs About Diagnoses: 127 833 Queries vs. 58 Benchmarks

Clinicians use language models very differently from how those models are tested. The authors of a new arXiv preprint analyzed 127 833 queries sent to an institutional LLM assistant over the eight months after its launch. The queries came from 6 342 physicians, advanced practice providers and nurses across 35 specialties. Documentation and administration accounted for 36.2% of all requests, and knowledge retrieval for another 28.9%. Diagnosis, the focus of most medical tests, got 3.7%.

How the study works

Readiness of clinical assistants for deployment is usually judged by benchmarks. Most of them are built from exam questions or pre-written clinical cases. Such a test predicts a model's behavior at work only as long as its tasks resemble real ones. How closely they resemble them has rarely been measured.

To measure it, the authors built RCQ-Map, a framework validated by clinicians. It draws on taxonomies of clinical questions and on taxonomies for LLM evaluation. Each query is annotated along five axes: task, intent, answerability, missing information and potential harm. The authors used the same tool to annotate 58 public benchmarks, taken from major evaluation suites and from frontier-model reports. The result is the Clinical AI Benchmark Atlas.

What the numbers show

The real workload breaks down as follows:

• documentation and administration: 36.2% of queries;
• knowledge retrieval: 28.9%;
• diagnosis: 3.7%;
• more than a third of queries could not be answered well as they were originally phrased.

The last point deserves separate attention. Real users often leave out context: they don't describe the patient, state the goal or attach data. An exam question is built differently, with a complete setup and an unambiguous answer. A model that solves such tasks brilliantly may do worse on a question where half the inputs are missing. Questions like that make up a noticeable share of the stream.

Where benchmarks diverge from practice

The median benchmark among the 58 contained not a single documentation query. Its task mix matched the real one by 31%. That is lower than a uniform distribution across task categories would give. In other words, a test assembled at random, with equal weight for every category, would on average sit closer to real life than the typical benchmark in use today.

The natural hypothesis is that sets billed as close to clinical practice should look better than exam-style ones. The data does not support it. Individually, benchmarks from such sets turned out to be no closer to real usage than those cited in frontier-model reports. The word "clinical" in a name does not guarantee resemblance to the actual workflow.

The limitations are also clear. This is data from one institutional system with one assistant, and the query mix at another hospital or in another country may differ. In addition, the authors measure how well the task mix matches, not the quality of the answers. A diagnosis benchmark can still be a valid measure of diagnostic ability. The problem is that it is used to judge a system's readiness for all of the work.

How to evaluate clinical assistants

For hospitals choosing or deploying an LLM assistant, the takeaway is practical. Public scores should be treated as only part of the picture. The model should be tested on a sample of the hospital's own queries, where documentation, administration and reference lookups dominate. Incomplete and ambiguous questions should be tested separately, along with whether the system asks for clarification instead of giving a confident answer. Potential harm can also conveniently be assessed for each task type.

For developers and benchmark authors, the signal is the same: build test sets with the real distribution of tasks in mind. The atlas the authors provide makes it possible to check which areas of practice remain unmeasured. As long as there are many of them, a high score on an exam-style test says less about how clinical AI performs at work than it seems.