Reasoning Models Think 20 Times Longer Than Needed on 2+3, Yet Slack Off on Hard Problems

Language models have learned to reason before answering, and a question familiar to every engineer immediately arose: how much time should be spent on a given task? Sherlock Holmes read a client's biography from their hands at a glance, yet for a tangled case he asked for three pipes and fifty minutes of silence. His key skill was knowing in advance what each problem required. Models are still poor at this.

Overthinking the Simple

In December 2024, researchers at Tencent AI Lab asked reasoning models an elementary question: "what is 2+3?" QwQ-32B-Preview found the correct answer in 39 tokens, but did not stop: it solved the problem another twelve times in different ways and spent 901 tokens. On average, new-generation models used almost twenty times more tokens than ordinary ones to arrive at the same result.

It is not only about money. A study by Meta and the Hebrew University, presented at ICLR 2026, showed that one model's shortest reasoning chain can be 34.5% more accurate than its longest. Accuracy grows with length up to a certain threshold and then falls, and the graph resembles an inverted letter U. Anthropic researchers added a distracting detail to a simple fruit-counting problem, and the longer Claude Opus 4 and DeepSeek R1 thought, the more often they erred. Sometimes a model finds the right answer and then talks itself out of it.

Underthinking the Complex

The mirror-image problem appeared when developers began teaching models to economize. In February 2026, Anthropic made adaptive thinking the default mode for Claude Opus 4.6: the model decides for itself whether to think about a step. In spring, programmers began complaining that Claude Code had become superficial: it invented Stripe API versions, commit hashes and nonexistent packages. Boris Cherny, the creator of Claude Code, analyzed the telemetry and found that on the steps where the model fabricated facts, it had spent not a single token on thinking. Where it thought, it answered correctly. This is the classic bat-and-ball problem: the fast system produces a plausible answer, while the slow one does not engage because the dispatcher deemed checking unnecessary.

Training a model to choose a mode is hard. First, the data for fast and reasoning modes differ greatly, and careless mixing yields "mediocre in both directions," as Junyang Lin of Qwen put it. Second, when training on rewards, the model quickly gets stuck in one mode that brings more reward on average.

What Is Being Tried

The Thinkless method teaches a model to begin its answer with the word "short" or "think," separating the reward for mode choice from the reward for quality. On algebra problems, reasoning was triggered in one problem out of four, 2.6 times fewer tokens were used, and accuracy fell by less than one percent. AdaptThink from Tsinghua forces answers without reasoning on half of the examples. A model with 1.5 billion parameters began using 45–65% fewer tokens and slightly gained in accuracy.

The industry has resolved the choice in different ways. Anthropic started with a manual budget in Claude 3.7 Sonnet, moved to an adaptive mode, and in the 5.x versions thinking cannot be turned off, leaving only the effort level. Alibaba built the hybrid Qwen3, abandoned it and split into Instruct and Thinking, then returned to a hybrid in Qwen3.5. OpenAI put an invisible router into GPT-5, which broke on launch day, and by autumn 2026 replaced Auto with a slider with levels from Instant to Extra High. DeepSeek merged R1 and V3 into one model with Non-Think, Think High and Think Max modes.

What to Do in Practice

If you are building a product on reasoning models, do not rely blindly on automatic depth selection. For tasks where the cost of error is high (API calls, code work, factual accuracy), fix a high effort level and verify that the model actually reasons on critical steps. For simple queries, on the contrary, set the fast mode: this saves tokens and time and also avoids needless doubt. And measure both skews, because a model can err from an excess of thinking as well as from a lack of it.