Apple and CMU Speed Up AI Agents 10x by Replacing Reasoning with Selection

Researchers at Apple and Carnegie Mellon University have proposed a way to speed up LLM agents by roughly 10x. Instead of writing out extended reasoning before each action, the model picks its next step from a set of options, as in a multiple-choice test. The Chinese outlet Jiqizhixin reported the work. The "10x" figure comes from the headline of the publication: the available excerpt contains neither the name of the method nor any benchmark tables, so the claim cannot be verified yet.

Where the agent loses time

A typical agent follows the ReAct scheme: observation, thought, action. Before every move, the model generates free-form text. It analyzes the current situation, weighs the evidence it has gathered, and only then decides what to do next. This is the core of the paradigm, and it is what gives agents their flexibility. But that flexibility has a price.

Tokens are generated sequentially, one after another. If the reasoning runs to hundreds of tokens while the action itself fits in about ten, most of the time and money goes to "thinking." On a task of twenty to thirty steps, these losses add up, and response latency grows almost linearly with the number of steps.

What Apple and CMU propose

The idea from the headline is to turn generation into a selection task. The model does not compose reasoning from scratch. It receives options and points to the suitable one. The output shrinks from hundreds of tokens to a few. That explains where an order-of-magnitude speedup could come from. But this is my interpretation of the mechanics: the available text does not disclose the details.

Here is what the published excerpt does tell us:

• authors: researchers at Apple and Carnegie Mellon University;
• problem: free-form reasoning before each action in ReAct creates noticeable inference overhead;
• solution: replacing free-form "thinking" generation with a choose-among-options format;
• claimed result: agent inference runs roughly 10x faster;
• not disclosed: the name of the method, the set of tasks, the models, and the effect on the share of tasks solved successfully.

The last point is the most important. Any compression of reasoning risks hurting quality, especially on long tasks, where an early error drags down the entire chain.

What to check first

Once you have the full paper, look for answers to several questions. Where do the options come from? If a large model also generates the candidates, part of the savings simply moves to another stage. How does the method behave in open-ended environments, where the action space does not reduce to a short list? What baselines did the authors compare against: full ReAct, short chain-of-thought, or other accelerators such as speculative decoding and caching?

Teams building agents can already estimate what share of the output tokens in their traces goes to reasoning. If it is more than half, a selection format or any other compression of the "thought" step could deliver a noticeable gain in latency and per-call cost. If reasoning makes up little of the traces, don't expect a tenfold effect.

The topic makes sense for Apple: cheap agent steps matter where compute is scarce. That is my assumption; the text offers no confirmation. For now, we have a claim of a strong result with no visible evidence. How seriously to take it can be judged after reading the paper itself, above all its accuracy tables on agent benchmarks.