ThinkJev Makes AI Think Only About Hard Questions: 28% Fewer Tokens

When a language model is embedded in a service, its answer is often read by a program rather than a person. For example: which queue to route a request to, whether a message violates the rules, whether an extracted record meets a condition. Researchers at the Chinese company Fujian Bosssoft call this a typed decision: the answer is a single letter from a closed set of options, and each option has a probability that code can rely on. Their system, ThinkJev, decides when such a model can answer instantly and when it needs to think. The headline number: on hard tasks, reasoning stops earlier and uses 28% fewer tokens, while accuracy drops from 82.7% to 81.6%.

Two speeds in one model

Anyone who has built such systems knows the problem. A fast answer takes one pass through the network: about 100 ms to process the prompt and about 53 ms to read the answer for Qwen3.5-4B on a single GPU. But on multi-step tasks it often gets things wrong. The authors built their own task set: date addition, counting under conditions, rule chains, nested negations, arithmetic, logic grids, graph reachability, scheduling and chess puzzles, in English and Chinese. Here Qwen3.5-27B gets 52.3% right with a fast answer (4B: 40.3%) and 87.5% after reasoning. But reasoning stretches into seconds.

The trick is that Qwen3.5 is a hybrid model: it can answer immediately or after a block of reasoning. In both modes the prompt is identical up to that block, and the answer is read from the same place: the probabilities of the letters A, B, C and so on. So one model provides two "systems":

• system 1 reads the letters after an empty reasoning block, that is, immediately;
• system 2 reads the same letters after a generated "thought".

The model processes the shared part of the prompt once. The fast answer is read from a copy of the cache, while reasoning continues on the original. The copy is necessary because Qwen3.5 mixes full-attention layers with Gated DeltaNet layers, and their state cannot be rolled back. There is no re-processing of the prompt when a task is escalated to system 2. Both systems produce probabilities in the same form, and the code receives them the same way. The answer text is not parsed, so there are no invalid answers.

An error guarantee and an answer at any budget

First, you have to decide which tasks can skip reasoning. The confidence threshold is chosen on a labeled sample using the Learn-then-Test method. It guarantees that the share of errors among accepted fast answers will not exceed a set target. Tasks of the same type from a single template count as one example, so duplicates do not inflate the guarantee. To remove the model's habit of favoring certain positions in the list, each option is read several times with the list cyclically shifted, and the results are averaged.

Results at a 5% target on 8359 test decisions:

• Qwen3.5-27B answers 24.5% of tasks on its own, with an actual error of 3.9%;
• for the smaller Qwen3.5-4B and 9B, this share is noticeably more modest, and at the strictest targets they take no decisions on themselves at all;
• for Qwen3.8-27B at the loosest target, the test error is slightly above the target.

Next comes the question of reasoning length. The letters can be read after any number of "thought" tokens: reasoning is cut off, an early-stop phrase that Qwen provides is appended, and the probabilities are read out. A single "thought" yields an answer at any budget, and a zero budget is system 1. The "anytime" policy simply stops at the first checkpoint where the answer is confident. For the remaining tasks, this gives 81.6% accuracy versus 82.7% with full reasoning, while using 72% of the tokens.

There is also an awkward finding. If you decide whether to think at all based solely on system 1's confidence, the savings are smaller. The tasks where the fast answer is most doubtful are not necessarily the ones that reasoning fixes. Predicting where the model will err and predicting where a thought will help are different problems.

What this means in practice

On the public tasks of JevBench, an independent benchmark for typed-decision services, Qwen3.5-27B thinks in 29.4% of cases. Its accuracy is no different from that of two commercial APIs with reasoning, though the authors call this comparison preliminary. The name refers to the request format of TypeSafe's commercial model Jev: the ThinkJev server accepts such requests, so its SDK can talk to a local model. The project is not affiliated with TypeSafe, and Jev itself was not tested.

For engineers, the takeaway is this. If you already run an open hybrid model, you can get "fast or think" routing without fine-tuning, with an error bound on the fast branch and without a second model. One limitation: the method works only for closed sets of options, not for free text. The code and data generators are open on GitHub (ding-xiaolu/ThinkJev).