Zyphra releases first MoE diffusion model, with a 7.7x inference speedup

Zyphra has released ZAYA1-8B-Diffusion-Preview, the first MoE diffusion model successfully converted from an autoregressive language model. The model shows that such a conversion is possible without quality loss, while inference gets 7.7x faster, a significant jump in performance.

What happened

Language models usually work autoregressively: they generate words one after another. That is slow, because each step depends on the previous one, so processing cannot be parallelized.

Zyphra converted ZAYA, a Mixture of Experts (MoE) model that selects different sub-networks of neurons for different inputs, into a discrete diffusion model. Diffusion generates text on a different logic: the model takes a noisy representation and refines it step by step, processing several layers of computation in parallel.

The idea is not new, since diffusion works well for images and text. But converting an MoE architecture from the autoregressive paradigm to a diffusion one while preserving quality is something earlier attempts did not manage this cleanly.

Why it is faster

The key is which GPU resource each mode uses. Two concepts matter here:

• Memory-bandwidth-bound tasks: they read a lot of data from memory and process little. Autoregressive generation is the classic example: you hold the whole context, reload it for every token, add the new token, and generate the next one
• Compute-bound tasks: they read data once and process it many times. Diffusion runs several cleanup iterations over the same tensor, and each iteration requires a full pass through the neural network
• GPU architecture: modern GPUs are growing faster in FLOPS (compute power) than in memory bandwidth. You have plenty of compute cores, but they often sit waiting for memory to deliver data

Moving ZAYA from memory-bound to compute-bound means the GPU's compute cores run closer to peak load. That is where the 7.7x speedup comes from.

Metrics held up

Zyphra tested the quality of the diffusion version against the original autoregressive ZAYA1-8B. Scores stayed at the same level: the model lost no ability to generate text, recognize context or follow instructions. That is not a given, since something usually drops when switching paradigms. Here, nothing did.

The result means the diffusion approach and the MoE architecture are compatible, and reconversion does not destroy the knowledge the model accumulated during training.

What it means

MoE diffusion models are moving out of the lab and into practical tools. For companies, this means they can take an existing MoE model and get a 7-8x inference speedup without retraining and without new GPUs.