CToM teaches LLMs to predict user replies and lifts accuracy to 79.6%

Researchers have published a method on arXiv called Collaborative Theory of Mind (CToM), which trains language models to work better with humans in long dialogues. The model first writes a draft, then predicts how the user will respond, and only then refines its answer. On a collaborative math reasoning task, final accuracy rose from 74.1% to 79.6% compared with plain GRPO. In coding and writing, with the same reward, users had to reply 8% and 17% less often, respectively.

Where collaboration breaks down

The authors start from an observation familiar to anyone who has done something more complex with an LLM than a single request. Models are strong at isolated tasks but behave awkwardly in multi-turn dialogue. They rush into solving the problem without finding out what the person actually needs. When the user corrects them, they do not revisit their earlier assumptions and keep building on a faulty foundation. Standard reinforcement learning on the final outcome does not fix this: the model gets no signal about what the human thinks of its move.

How CToM works

The proposed loop has three steps: a draft response, a prediction of the user's reaction, and an internal revision based on that prediction. The user sees only the final result. For training, the real user reply to each draft serves as the target for predicting the next turn, and the assistant is trained to minimize the negative log-likelihood of that reply. This objective is combined with GRPO (group-relative policy optimization). As a result, the model optimizes two things at once: the quality of the solution and the efficiency of the interaction, meaning the number of turns it took to reach the result.

The abstract does not say whether the user replies were real or generated by a simulator. That matters for how well the result transfers to real products, and the answer will have to come from the full text of the paper.

The numbers

The method was tested on math reasoning, coding, and writing. According to the authors, CToM outperforms all baselines on collaboration quality. Key results:

• math: final accuracy of 79.6% versus 74.1% for naive GRPO, a gain of 5.5 percentage points;
• coding: with the same reward, adding supervision on prediction cuts the number of user replies by 8%;
• writing: a 17% reduction in the number of replies, with quality improving at the same time;
• behavioral analysis: after training, predictions of user reactions become more accurate.

The last point matters most because it points to a mechanism. The authors found that more accurate predictions are associated with a larger quality gain from self-revision. That suggests the improvement comes from the model's ability to model its conversation partner, not from side effects of training. The finding is still correlational, and treating it as proven causation would be going too far.

What developers should do with this

For teams building assistants and coding agents, the idea is practical. User replies after each response are already in the logs, and they provide a ready-made training target for next-turn prediction. The metric "how many times the person had to correct the model" is worth tracking alongside accuracy: saving 8–17% of turns in a long session is noticeable in real work.

There are limitations. A draft plus a prediction plus a revision means extra tokens and latency, and the abstract does not discuss inference cost. A 5.5 percentage point gain on math is modest, and the work remains a first-version preprint with no independent verification. If the reaction prediction is wrong, the revision can steer the answer off course. Before adopting the method, it makes sense to run it on your own dialogues and compare the number of turns and the final quality against a baseline model.