Diffusion LLMs Spend Over Three Times as Many Passes on a Story as on Grammar

Diffusion language models are attractive because they produce several tokens per forward pass and can write faster than autoregressive ones. A new analysis of two such models across 10 tasks shows that their speed fluctuates. In DiffusionGemma, a token for writing a story takes more than three times as many passes as a token for correcting grammar. Two answers of the same length in tokens can cost entirely different amounts of compute.

Why copying is cheaper than composing

The mechanism is this: during generation, a diffusion model decides on its own how many passes to spend. If part of the text is already clear, it locks in many tokens at once. If not, it runs more iterations and refines.

The authors compared costs across tasks and found a clear pattern. Text copied from the input or only lightly edited is cheap. Open-ended writing is expensive. Grammar correction consists almost entirely of material the model already has, while a story has to be invented from scratch.

The researchers attribute the gap to two properties of the output text:

• the share of new material relative to the input;
• the rarity of words within a given task.

The more new and unusual content an answer contains, the more passes the model spends on each token.

The same logic applies within a single answer when the model reasons first. When the final answer repeats the model's own chain of reasoning, it is cheap. When it adds something new, the cost goes up. In other words, the compute advantage of diffusion depends on how much of the output is already determined.

A tokenization tax in multilingual generation

A separate finding concerns languages. Those that the tokenizer splits into many pieces need more passes per token. They already pay more for sheer volume, since text in such a language contains more tokens. The result is a double burden, and the difference in serving cost between languages turns out to be even larger than sequence lengths suggest.

For companies that sell one model across dozens of markets, this matters in practice. Counting by tokens underestimates the real cost for heavily tokenized languages if inference is built on diffusion.

Decoding speed as a signal of correctness

The most practical part of the work concerns quality. On the same math problem, wrong answers take more passes per token than correct ones. The model seems to hesitate where it is not sure.

The authors used this as a signal: they weighted candidate answers by decoding speed. This approach beat simple majority voting by up to 4.2 points. It requires no additional compute, because the number of passes is already known during ordinary generation.

Key metrics of the study:

• models: two diffusion models, including DiffusionGemma;
• benchmark set: 10 tasks;
• story vs. grammar correction: more than 3 times as many passes per token;
• cost factors: novelty relative to the input and word rarity within the task;
• gain from speed weighting over majority voting: up to 4.2 points at no additional cost.

What to change in inference and planning

First, you cannot budget diffusion inference by output length. You need to estimate the task type. Editing, paraphrasing, formatting, and extracting structured data from the input fall into the cheap zone. Generating long creative texts falls into the expensive one, and a service with a broad workload profile will see a wide spread in latency and cost.

Second, the number of passes can be logged as telemetry. A spike on a particular request points to a high risk of error, and it makes sense to send such answers for rechecking, resampling, or human review. For self-consistency tasks, where several candidates are generated, speed weighting delivers a gain at no extra cost.

Third, when pricing multilingual products, you need to account for both effects: the number of tokens and the number of passes per token. Otherwise, languages with heavy tokenization will quietly become unprofitable.

The study's limitation is clear: it covers two models and 10 tasks. But the direction is clear. Diffusion LLMs win where a significant part of the answer is predictable, and the decoding process itself carries information that almost no one uses yet.