Alibaba Qwen Unveils TRACE: FP4 Rollout Runs Up to 5.4x Faster

The Alibaba Qwen team has introduced TRACE, a reinforcement learning method for large models that moves the Rollout phase to FP4: weights, activations (W/A) and the KV-cache alike. According to the authors, Rollout runs up to 5.4x faster than with BF16, and training stays stable. Stability matters more here than speed, because many teams can run generation in four bits, but far fewer can keep RL from falling apart while doing it.

Why Rollout became the most expensive part of RL

In RL for language models, the loop works like this: the model generates many responses, often long reasoning chains, those responses are scored, and the weights are updated based on the resulting trajectories. Generation proceeds token by token and is bound by memory bandwidth, while the KV-cache grows with response length. The longer the reasoning and the more samples per prompt, the more time and VRAM go to Rollout rather than to the gradient descent step.

Hence the obvious urge to compress this stage. In theory, moving from BF16 to FP4 cuts both the amount of computation and memory use. Fewer bytes per weight and per cache element means more parallel requests per card and less waiting on data.

Where FP4 breaks training

The price of this compression is a coarse numerical grid. Four bits hold only 16 values, so two very close quantities can easily fall on opposite sides of a quantization boundary and land on different code points. If the trainer and the Rollout engine compute at different precision, they produce noticeably different results from nearly identical inputs. A tiny numerical difference turns into a gap between what the model "generated" and what it "thinks" when the weights are updated.

This is dangerous for RL. The algorithm assumes the trajectories were produced by a policy close to the one being trained. When the generator and the trainer diverge, the gradient starts to rely on data that the current model would have produced differently. This usually shows up as noise, drift and unstable training. With FP4 the problem is sharper than with FP8, because there are more boundary effects and less precision headroom. This is the problem TRACE sets out to solve.

What we know about TRACE

Key parameters of the work:

• Developer: the Qwen team (Alibaba).
• Scope: the Rollout stage of RL training for large models.
• Format: FP4 for weights and activations (W/A) plus FP4 for the KV-cache.
• Baseline: BF16.
• Claimed effect: Rollout speedup of up to 5.4x.
• Goal: preserve training stability at low generation precision.

"Up to 5.4x" is a maximum, not an average result. The actual gain depends on generation length, batch size, model architecture and hardware. The longer the context and the heavier the KV-cache, the more noticeable the benefit of cache quantization, because the cache often becomes the bottleneck on long reasoning runs.

What teams running RL on their own clusters should check

For anyone training reasoning models in-house, the main takeaway is that Rollout deserves to be treated as a separate optimization problem, not a side process. If generation takes most of the step time, reducing precision pays off more than tuning the optimizer.

Before switching to FP4, measure the divergence between token probabilities in the trainer and in the Rollout engine. If it is already noticeable on BF16 or FP8, it will grow at four bits. Separately, check quality on long trajectories: KV-cache error accumulates with response length.

Finally, the gain depends on hardware support. Native FP4 is available in newer accelerator generations, such as the Blackwell architecture, while on older cards the 4-bit path will not deliver the same speedup. For labs with a limited GPU fleet, work like TRACE is interesting because it allows more RL iterations on the same hardware. The next question is how well the result holds up on models of other sizes and on other tasks, from math to code.