DeLS-Spec speeds up LLM inference without retraining models

Researchers have proposed DeLS-Spec, a new approach to speeding up text generation in language models. It is the first method to deliver a significant speedup without retraining the models themselves, which substantially lowers the barrier to adopting such optimizations.

How speculative decoding works

Speculative decoding is a technique that speeds up LLM inference by generating several tokens in parallel instead of one at a time. A small drafter model predicts candidate tokens, and the main model checks their correctness in a single pass. If the prediction matches, the tokens are accepted; if not, the process repeats.

More advanced methods, such as DFlash, generate an entire block of tokens in one pass, speeding things up even further. But this creates a problem: tokens within a block are usually predicted independently, without accounting for causal dependencies between them.

The problem with existing methods

Other recent methods, such as Domino and DSpark, tried to add causality to block-parallel drafting by accounting for dependencies between tokens in a block. But that required training the drafter entirely from scratch, which is expensive and demands enormous computing resources. Organizations without petabytes of data and large-scale GPU clusters cannot afford such retraining.

What is new in DeLS-Spec

DeLS-Spec solves this problem by splitting the work between two experts. The method uses an existing DFlash model as a "long-context expert" and adds a lightweight local head as a "short-context expert."

The key advantage: the local head can be trained completely independently, without joint training with the main model or the DFlash backbone. This sharply cuts computational costs. The local head is a small network that looks only at recent context and learns to predict tokens while accounting for causal dependencies.

• The method was applied to Qwen3 models
• Training requires minimal resources: only the lightweight adapter is trained
• At inference time, logits from the two experts are combined
• The local head does not depend on a specific DFlash checkpoint version
• Both speed and accepted token length improved on benchmarks

Test results

On Qwen3 benchmarks, DeLS-Spec improved both execution speed (speedup) and average acceptance length compared with DFlash. This held across three task categories: math, programming and dialogue systems.

Why it matters: longer runs of accepted tokens mean the drafter predicts more accurately, so the system can generate more text per iteration, which directly affects throughput.

Why this matters for the industry

Speeding up LLM inference is not just a question of speed but also of economics. Training a small adapter is cheaper than retraining the whole drafter or the main model. That makes optimization methods more accessible to organizations that lack the resources for large-scale retraining.

As LLMs grow more complex and power-hungry, incremental improvements in inference efficiency add up to meaningful gains in speed and cost. DeLS-Spec illustrates an important trend: the best results often come not from retraining large models but from combining existing components elegantly.

What this means

This approach opens the way to more accessible LLM optimization in organizations of all sizes, allowing even small teams to achieve speedups without huge compute budgets.