九章云极 Alaya-DFlash2 speeds up GLM-5.3 decoding by 3.41x

Cloud provider 九章云极 (DataCanvas) has unveiled Alaya-DFlash2, a decoding acceleration system, and claims a 3.41x increase in GLM-5.3 generation speed. It targets a problem familiar to anyone who has handed a model a long document: it reads fast but answers slowly.

Where long context slows things down

The wait happens on output, not on input. The model swallows a 32-thousand-token document in a single pass, and the first word of the answer appears almost immediately. Then step-by-step generation begins: the main model takes one step forward, emits one token, and repeats the whole process. According to the source, it has to reprocess the entire context at every step. Technically, this is mostly reading the KV-cache, but the conclusion is the same: the cost of a single step grows with context length. The thicker the document, the slower the answer drags on.

How speculative decoding works

Alaya-DFlash2 is built on a scheme that has become the standard for inference acceleration in recent years. A small draft model guesses several tokens ahead at once. The large model verifies the whole chain in a single pass. Matching tokens are accepted immediately, the chain is cut off at the first mismatch, and generation continues from the correct token. If verification is implemented correctly, the final text is identical to what the main model would have produced on its own. The only gain is speed.

Everything comes down to two quantities: how many tokens in a row the main model accepts from the draft on average, and how much the draft itself costs. On long context, the second quantity becomes a problem, because the draft model also needs to "see" the whole document. This is exactly where the breakthrough is claimed. How the second version of DFlash actually works is not disclosed in the available part of the material.

What the announcement tells us

• Solution: Alaya-DFlash2 from 九章云极
• Target model: GLM-5.3
• Claimed decoding speedup: 3.41×
• Scenario: long texts; the source gives a 32k-token document as an example
• Method: speculative decoding with a draft model and batch verification by the main model

The hardware, batch size, comparison baseline, average accepted chain length, and quality-check results are not specified. Until they are, the 3.41× figure should be treated as a marketing maximum, not a guarantee.

How to read the number and what to check

Multi-fold speedups from speculative decoding are usually achieved under light parallel load. As batch size grows, the GPU is already busy with useful work, and the gain from guessing shrinks. The result also depends heavily on the type of text: code and structured answers are predicted better than free-form prose. So "3.41x" with no stated load regime describes a ceiling, not a typical case.

Anyone running GLM in production on long documents (contracts, reports, correspondence, codebases) can take three steps:

• Measure generation speed on your own traffic with realistic context lengths, not on short benchmarks.
• Compare the gain for a single user against peak concurrency. For a service with a queue, the second number is the one that matters.
• Run a sample of responses and confirm that quality has not dropped relative to baseline decoding.

For the market, the fact itself matters more: competition over long context is shifting from window size to the price and speed of responses within it. Providers that squeeze decoding harder in software will be able to serve more tokens from the same GPU fleet. That is a direct saving wherever accelerators are scarce.