Seq-Drifting Generates Text in a Single Pass: Perplexity of 36.01 on LM1B

Diffusion language models generate tokens in parallel, but good text still takes many sampling steps. The authors of Seq-Drifting try to remove those steps entirely: their generator takes Gaussian noise and produces the whole sequence in a single forward pass. On the LM1B benchmark, this one-step mode yields a generative perplexity of 36.01. The preprint is on arXiv (2610.12879), and the code is published on GitHub.

How the drifting field works

In standard diffusion LMs, the training signal for each position comes directly from the token at that position in the training sequence. The model learns to reconstruct a specific layer of the data, and quality comes through iteration: each step refines the previous one.

Seq-Drifting works differently. The method applies drifting to sequences of continuous token representations and directly trains a mapping from noise to text. To do this, it builds a drifting field in which positional targets depend on the context generated so far, rather than being fixed in advance by a single reference token. Two forces act at each position. Attraction pulls the output toward a set of contextually plausible token representations. Repulsion solves two problems at once: it keeps samples from collapsing into each other and suppresses repetition within a single sequence.

The field is needed only during training. At inference, the generator runs no iterations and emits all tokens jointly in one pass.

What the experiments show

The main result is on LM1B, where one-step generation reaches a generative perplexity of 36.01. According to the authors, the model is competitive in both unconditional and conditional generation. They also tested whether the approach works beyond free-form writing: the experiments cover translation, summarization, and reasoning tasks.

Key parameters of the work:

• Inference mode: one forward pass, all tokens in parallel.
• Generator input: Gaussian noise; output: a sequence of continuous token representations.
• Training signal: a drifting field with attraction toward plausible tokens and repulsion against collapse and repetition.
• Metric: generative perplexity of 36.01 on LM1B.
• Tasks: unconditional and conditional generation, translation, summarization, reasoning.

Several important things are not visible from the abstract. There is no numerical comparison with multi-step diffusion baselines, no latency measurements, and no indication of which model was used to compute perplexity. Generative perplexity is also sensitive to the choice of the scoring model and does not, on its own, show text diversity. These details will have to be checked in the full paper.

Who this is useful for

The main practical interest here is inference cost. If the quality of a diffusion LM depends on the number of steps, each step multiplies the cost of calling the model. A one-step generator removes that multiplier in principle, and for tasks like translation or summarization with strict latency requirements, that is a serious argument.

For now, this is a research result. LM1B consists of short sentences, and testing on long contexts, larger models, and instruction-following tasks is still ahead. Anyone working with diffusion LMs should run the open repository and check three things: how quality changes with sequence length, whether diversity holds up in a single pass, and how much the result depends on the repulsion parameters. If the answers turn out favorable, Seq-Drifting will become a notable argument in the debate over whether language models need iteration at all.