TraceSpec raises DSpark throughput for LLM agents by 12.5–22.6%

A team from Renmin University of China and Huawei has presented TraceSpec, a pairing of an evaluation set and a training method for the draft model in speculative decoding for agents. The idea: an agent makes dozens of calls to the model over a growing history of reasoning, actions and tool responses, and the usual average token acceptance metric lumps together very different decisions. TraceSpec breaks these decisions apart, then trains the draft model where it most often disagrees with the target model. On Qwen3 at 4B, 8B and 14B, at two temperatures, throughput in replay mode rose 12.5–22.6% over the DSpark baseline.

Why the average metric misleads

The authors first measured how speculative decoding behaves during live agent work. They used two scenarios: BrowseComp-Plus (finding answers across web documents) and SWE-bench Verified (fixing real bugs in repositories). For DSpark with Qwen3-8B, average accepted length in browsing grew from 3.633 tokens per round at contexts up to 2k to 5.479 at 16k–32k contexts. Over the same range, throughput fell from 82.73 to 22.54 tokens per second. In software engineering, it dropped from 73.83 to 28.19 tokens per second.

On its face, this looks like "long context helps the draft model." But when the researchers compared comparable late-stage browsing actions within the same tasks, acceptance actually declined. The rise in the average comes from the mix of calls: long histories contain more short, predictable tool requests. In the SWE scenario the curve is not monotonic at all: accepted length peaks at 8k–16k and falls at 16k–32k for all three methods tested — DFlash, EAGLE-3 and DSpark.

How the corpus and training work

For a fair comparison, the authors built TraceSpec-Traj: 10 000 training and 2 500 test states, split evenly between web research and software engineering. A state is the history up to the assistant's next turn, together with its phase (planning, action, reflection, final answer), prefix length and position in the trajectory. The corpus covers all 40 combinations of two domains, five context ranges and four phases. Trajectories were assembled from DeepResearch-traj, Nebius SWE-agent data and R2E-Gym collections; tasks in the training and test parts do not overlap. In replay, all five baseline methods receive identical saved prefixes.

TraceSpec itself keeps DSpark's architecture and distillation function. What changes is the data and its share in training. Instead of final answers, the draft model is trained on all intermediate turns of trajectories generated by the target model. For each state, the method estimates the acceptance gap, corrected for prefix: an early divergence cuts off the chain of consecutively accepted tokens, so it carries more weight. The estimate is smoothed across context, phase, task type and tool state. The prediction budget is fixed, and short tool requests and long reasoning yield different numbers of predictions, so sampling is normalized by the number of events.

What the experiments showed and where the limits are

The authors compared TraceSpec with five baseline methods across 108 configurations. In replay, it leads in all 24 per-phase comparisons. On generation throughput it beats DSpark in 15 of 18 settings, so the gain is not universal. Among the baselines, DSpark showed the longest accepted length in all 18 comparisons, and accepted length falls for every method when moving from temperature 0 to 1.

Ablations on Qwen3-4B show the contribution of individual components. Adaptive budget allocation raises accepted length by 3.3–10.0% compared with fixed training on full turns. Additional comparisons confirm the value of state conditioning and of accounting for consecutively accepted tokens.

The limitations are also visible. The main gain of 12.5–22.6% was measured in replay of saved prefixes, not in end-to-end episode latency. Only Qwen3 models and two scenarios, browsing and programming, were tested. The article excerpt cuts off at the analysis of episode cost, so an end-to-end gain in task time cannot be confirmed here. The practical takeaway for anyone serving agents: before fine-tuning a draft model, break down acceptance by phase and by position in the trajectory. The authors have published code and resources on GitHub in the Robot2050/TraceSpec repository.