MIT researchers teach AI to honestly say "I'm not sure" and hallucinate less

MIT researchers have proposed a way to make language models noticeably more honest: instead of just solving a problem, the model also estimates how confident it is in its own conclusion. That sounds like a small tweak, but it targets one of the most frustrating habits of today's reasoning models: they sound confident even when they arrived at an answer almost by guessing. The new method doesn't hurt answer quality.

Instead, it helps the model tell apart cases where it actually knows something from cases where it should admit uncertainty. The MIT CSAIL team puts the problem bluntly: today's strong models often act like the loudest person in the room. They answer with the same confidence whether their reasoning worked or they simply guessed.

The researchers trace the cause to the reinforcement learning setup now used to build reasoning abilities. In a typical setup, the model gets a reward for a correct answer and a penalty for a wrong one. There is almost nothing in between.

If the model stumbles onto the right result by chance, it gets the same reward as if it had carefully worked out the solution. Over time, this pushes the system to always answer and leaves no room for "I'm not sure." That is what MIT set out to fix with RLCR, short for Reinforcement Learning with Calibration Rewards.

Instead of a purely binary score, the researchers added another component to the reward function: the Brier score, a metric that compares stated confidence with actual accuracy. In practice, after its chain of reasoning, the model outputs both an answer and a numerical estimate of its own confidence. If it is overconfident and wrong, it gets penalized.

If it gives a correct answer but lowballs its confidence for no reason, that counts against it too. The authors claim this setup formally pursues two goals at once: high accuracy and good calibration, meaning a match between how confident the model says it is and how often it is actually right. The experiments used a 7-billion-parameter model.

According to MIT, RLCR cut calibration error by up to 90 percent compared with standard reinforcement learning, while accuracy did not drop and even improved on some tests. The effect held not only on the tasks the model was trained on but also on new datasets, including six datasets it had never seen before. The researchers also compared the method with post-hoc approaches, which try to estimate confidence after training using an external classifier.

RLCR came out ahead here as well: rather than bolting a cosmetic layer onto a finished model, it changes the system's behavior during training. What's more, the MIT team shows that standard RL not only fails to improve calibration but often makes it worse: the model gets more capable but also more overconfident. There is a practical payoff, too.

If a model generates several candidate answers, you can pick the one where it reports the highest confidence, or weight the candidates' votes by that estimate. According to the authors, this improves both accuracy and calibration as inference compute scales up. Another interesting finding: when the researchers trained separate classifiers on the model's outputs, explicit reasoning about its own uncertainty provided an additional useful signal, especially for smaller models.

In other words, the model's attempt to spell out what it knows and what it doesn't turns out to be a meaningful part of the prediction, not decoration. What does this mean in practice? If RLCR scales to larger commercial models, the industry gets a chance to reduce not only obvious errors but also dangerous errors disguised by a confident tone.

This matters most in fields like medicine, law, finance, and corporate analytics, where users need more than an answer: they need to know how far they can trust it. Rather than yet another filter on top of an already trained model, the MIT work offers a more fundamental idea: teach AI not only to find a solution but to honestly measure the limits of its own knowledge. That habit may turn out to be one of the most useful upgrades for the next generation of reasoning systems.