Google's DiffusionGemma writes text in blocks of 256 tokens at once, rather than one at a time

Almost every language model you use works the same way: it emits a token, appends it to the text, runs everything again, and emits the next one. This approach has two familiar drawbacks. First, when running locally for a single user, the bottleneck is memory rather than compute. To produce one token, the model must load all its weights every time, while the GPU’s compute units sit idle. Second, once something has been written, it cannot be rewritten. If the model reaches a dead end halfway through an answer, it cannot backtrack.

DiffusionGemma from Google DeepMind attempts to solve both problems at once. It is an experimental open model released under Apache 2.0 on June 10, 2026. Under the hood is the Gemma 4 26B architecture: a Mixture-of-Experts model with 26 billion parameters, of which 3.8 billion are activated per token—hence the A4B suffix.

How it writes

The model starts with a canvas of 256 placeholder tokens—essentially noise—and processes the entire block over several passes. Tokens it is confident about help determine the others, and the text gradually “develops,” like an image in Stable Diffusion. Here, however, words take the place of pixels.

If an answer is longer than 256 tokens, the model switches to a block-autoregressive mode. The completed canvas is stored in the KV cache using conventional causal attention, and a new canvas begins with access to everything that came before. Within each block, attention is bidirectional: any token can attend to any other token. This enables the main new capability—self-correction. If confidence in a token drops, the sampler can add noise to it again and let the model revise it. A conventional LLM cannot do that.

The practical effect is that the bottleneck moves from memory bandwidth to computation. The GPU gets genuinely parallel work instead of waiting between steps. That is where the speed advantage is expected to come from.

How to run it

The model accepts multiple input modalities: text, images of arbitrary size, and video, but not audio. Its output is always text. This is not an image generator carrying a diffusion label; diffusion is applied only to text. It supports system instructions and an adjustable thinking mode, enabled with enable_thinking=True, which outputs its reasoning in a separate block.

The hardware requirements are as follows. For the full model, Google recommends a GPU with more than 60 GB of memory, such as an A100 or H100. The quantized version fits in 18 GB of VRAM, so it can run on an RTX 4090 or 5090.

On the software side, transformers 5.11.0 or later is required; older versions simply do not recognize the model. It must be loaded through the dedicated DiffusionGemmaForBlockDiffusion class rather than the familiar AutoModelForCausalLM because it has its own denoising loop. AutoProcessor is used instead of a tokenizer, since the model also handles images and video. TextDiffusionStreamer lets you watch the process: it prints the canvas state after every step, showing the text assembling itself out of noise.

For production use, Google worked with the vLLM team to integrate iterative denoising into the standard engine. The vllm serve command supports long context—in the example, 262144 tokens—and the sampler is configured through hf-overrides, for example with entropy_bound.

What to do with it

One caveat up front: this is a research preview, not a finished product, and Google says so explicitly. Before using it with customers, review the Gemma acceptable-use policy. The material provides no real-world acceleration figures, so you will need to measure the latency gains on your own workloads.

Still, it is worth trying. If you build local assistants, run pipelines with a high volume of short generations, or simply track where LLM architecture is heading, text diffusion is becoming a practical tool that can run on a single gaming GPU. Start with TextDiffusionStreamer: five minutes spent watching the denoising process will explain the mechanics better than any article.