DCOCO speeds up LLM inference by up to 6.6x and beats rivals by 6–23%

Researchers have published DCOCO (DFlash Contextual Corrector) on arXiv, a method that addresses the main weakness of diffusion drafters in speculative decoding. It is a lightweight corrector that keeps the draft generation parallel while restoring dependencies between tokens within a block. Across four verifier models and ten benchmarks, it achieved the highest average throughput in every configuration tested. The maximum speedup was 6.6x relative to standard decoding. At 4–32 concurrent requests, the method outperforms the strongest competitor by 6–23%.

Where the parallel draft breaks down

Speculative decoding is simple in principle. A small drafter model proposes several tokens ahead, and a larger target model verifies them in a single parallel pass. Everything that matches the target model's own prediction is accepted. The longer the accepted run, the higher the speed.

Diffusion drafters made this scheme more attractive. They have a deep attention backbone with strong predictions, and they fill future positions in parallel, so they don't pay for sequential autoregression. But parallelism has a price. The token at the third position of a block is predicted without knowing which token landed at the second. The dependencies within a block, which arise naturally in autoregression, disappear.

In practice, each token looks plausible on its own, but neighboring tokens don't fit together. The verifier discards the tail of the block, and the length of the accepted draft drops. Part of the gain from parallel drafting is eaten away.

What DCOCO does

The authors did not go back to sequential generation. Instead of locking in a single leader for each position, DCOCO keeps several candidates. It then evaluates in parallel how compatible the candidates at neighboring positions are with each other, and selects tokens with that context in mind. According to the authors, the additional cost is small.

The key difference from existing correctors is that those work sequentially. They also restore dependencies, but they bring back into the pipeline exactly the sequential cost that the diffusion drafter had eliminated. According to the paper, at a load of 4–32 concurrent requests, such correctors fall below baseline DFlash with no correction at all.

Numbers from the abstract:

• 4 verifier models and 10 benchmarks in the tests;
• highest average throughput in every evaluated configuration;
• modes: greedy and stochastic decoding, with and without extended reasoning;
• speedup of up to 6.6x relative to standard decoding;
• an advantage of 6–23% over the strongest competitor at 4–32 concurrent requests.

It also matters that the result holds in modes with long reasoning chains. That is where generation is most expensive and where faster inference pays off the most.

What to watch for in deployment

This is a preprint, and all the figures come from the authors themselves. The abstract contains no independent verification and no data for specific models or hardware, so 6.6x should be read as a ceiling, not as the result to expect on your traffic.

A team running an LLM service should take two things from this work. First, speedup methods should be compared on more than a single request. A corrector that wins at batch size 1 can lose at 8–32 concurrent requests, and the paper shows this directly. Second, if you already use DFlash or a similar diffusion drafter, DCOCO looks like an add-on rather than a replacement for the whole scheme. It makes sense to run it on your own prompts and load profile: separately for short answers and for long reasoning, and separately for greedy and stochastic sampling. Track the average accepted block length and the resulting tokens per second on the GPU, not a single metric.