DeLS-Spec accelerated LLM inference without retraining models

Researchers proposed DeLS-Spec—a new method for accelerating text generation by language models. The approach for the first time allows achieving significant speedup without the need to retrain the models themselves, which substantially lowers the barriers to adopting such optimizations.

How Speculative Decoding Works

Speculative decoding is a technique that accelerates LLM inference by generating multiple tokens in parallel instead of one at a time. A small draft model predicts candidate tokens, and the main model verifies their correctness in a single pass. If the prediction matched—tokens are accepted; if not, the process repeats.

More advanced methods, such as DFlash, generate an entire block of tokens in a single pass, further accelerating the process. But this creates a problem: tokens within a block are usually predicted independently, without considering causal dependencies between them.

The Problem with Existing Methods

Other recent methods, such as Domino and DSpark, attempted to add causality to block-parallel drafting—to account for dependencies between tokens in a block. But this required training the drafter from scratch, which is expensive and requires enormous computational resources. Organizations without petabytes of data and massive GPU clusters cannot afford such retraining.

What's New in DeLS-Spec

DeLS-Spec solves this problem through division of roles between two experts. The method uses the existing DFlash model as a "long-context expert" and adds a lightweight local head as a "short-context expert."

The key advantage: the local head can be trained completely independently, without joint training with the main model or the DFlash backbone. This drastically reduces computational costs. The local head is a small network that looks only at recent contexts and learns to predict tokens taking into account causal dependencies.

• Method applied to Qwen3 models
• Training requires minimal costs—only a lightweight adapter is trained
• At inference, logits from two experts are combined
• Local head is independent of any particular DFlash checkpoint version
• Improvement in both execution speed and length of accepted tokens on benchmarks

Testing Results

On Qwen3 benchmarks, DeLS-Spec showed improvement in both execution speed (speedup) and average length of accepted tokens (average acceptance length) compared to DFlash. This holds true for three categories of tasks: mathematics, programming, and dialog systems.

Why this matters: longer sequences of accepted tokens mean that the drafter predicts more accurately and the system can generate more text in a single iteration, which directly affects throughput.

Why This Matters for Industry

Accelerating LLM inference is not just a matter of speed, but of economics. It is cheaper to train a small adapter than to retrain an entire drafter or base model. This makes optimization methods more accessible to organizations that do not have resources for large-scale retraining.

As LLMs become increasingly complex and energy-intensive, such incremental improvements in inference efficiency accumulate and turn into significant gains in speed and cost. DeLS-Spec demonstrates an important trend: better results are often achieved not through retraining large models, but through elegant combination of existing components.

What This Means

This approach opens paths for more accessible LLM optimization in organizations of all sizes, enabling even small teams to achieve acceleration without massive computational budgets.