ReaDy speeds up hybrid LLM decoding 3.16x on a single B200

Researchers have introduced ReaDy, a speculative decoding method for hybrid linear-attention models. Across seven benchmarks covering math, code generation and dialogue, it speeds up generation by an average of 3.16x compared with standard autoregressive decoding. The tests ran on Qwen3.8-27B and a single NVIDIA B200. ReaDy beats the earlier DSpark approach consistently: it is better on all seven tests, by an average of 7.97% in throughput and 8.50% in accepted sequence length.

Why the drafter looks in the wrong place

Speculative decoding is simple in principle. A lightweight drafter predicts several tokens ahead, and the main model verifies them in a single pass and accepts the correct prefix. The longer the accepted chain, the fewer expensive steps the large model has to take.

Many current drafters rely on the target model's hidden features. The ReaDy authors point to a weakness in this approach: those features carry their own "contextual emphasis," which does not necessarily match what drafter positions further down the chain need. One part of the history matters more for the first token, while the fourth may need another. A single set of features serves these needs unevenly.

How ReaDy works

Hybrid linear models store compact recurrent states that accumulate historical associations. These states are a convenient source of extra context for speculative decoding: the target model has already computed them, and they take up little memory. According to the authors, ReaDy is the first to use these states to retrieve context for a specific drafter position.

The mechanics:

• Each drafter position gets its own learnable query.
• The query reads additional context directly from the target model's committed recurrent states.
• A lightweight state reader adds what it finds to the drafter's representations through residual updates.
• Low-rank state reading and fused operators keep the extra compute and I/O load low.

ReaDy builds on DSpark's parallel backbone and sequential prediction head. Adaptation takes two stages. In ablations, the authors tested reader capacity, state sources and where to insert the residual updates. The abstract gives no specific results from these ablations.

The numbers

• Benchmarks: seven, covering math, code and dialogue.
• Model and hardware: Qwen3.8-27B, a single NVIDIA B200.
• Average speedup over autoregressive decoding: 3.16x.
• Versus DSpark: throughput +7.97%, acceptance length +8.50% on average.
• ReaDy beats DSpark on each of the seven tests.

Note how the two metrics relate. An 8.50% gain in acceptance length against a 7.97% gain in throughput means that nearly all of the improvement comes from more accurate drafts, while the extra compute for reading states eats only a small share. This is consistent with the authors' emphasis on low-rank reading and fused operators.

What to check before deploying

The method is tied to a specific architecture. It requires recurrent states, so it suits hybrid linear models but does not apply, in this form, to classic full-attention transformers. If your production stack runs on HLA models, ReaDy belongs on your shortlist. The code is open on GitHub (StartLuxLabs/ReaDy), and the two-stage adaptation means you need to fine-tune the drafter for your own target model rather than use it off the shelf.

The description omits data that matters for real-world operation. It is unclear how the method behaves with large batches, long contexts or limited memory. All measurements come from a single B200, and the comparison with DSpark uses a single set of tests. Before building 3.16x into your inference cost estimates, run the method on your own traffic: your mix of code, math and dialogue may differ, and the speed of speculative methods depends heavily on how predictable the text is.