Google unveils DiffusionGemma, a model that generates text a different way

Google has released DiffusionGemma, an experimental model that takes a fundamentally different approach to text generation than the models behind today's chatbots.

How chatbots work today

Modern text AI models (GPT, Claude, Gemini) use the transformer architecture. The principle is simple: the model processes text from left to right and predicts the next word, one at a time. It is like writing in the dark, one character after another, with no chance to redo earlier steps.

This scheme works flawlessly and has become the industry standard over the past three years. But it has a built-in limitation: the model cannot go back and revisit its decision. After each token is emitted, the context moves only forward. If the model has already made a mistake at the 50th token, it cannot fix it.

The DiffusionGemma diffusion approach

DiffusionGemma uses a completely different logic: a diffusion process that Google borrowed from image generation. There it has long delivered excellent results (DALL-E, Midjourney and Stable Diffusion work exactly this way).

In a diffusion model, everything happens iteratively rather than sequentially. The model starts with random noise across the entire length of the text at once. On the first iteration, rough outlines of meaning appear. On the second, syntactic structures are refined. On the third, stylistic details are added. It resembles an artist's work: first a charcoal sketch, then the main colors, then light and shadow, then the final strokes.

The key difference: at every step of diffusion generation, the model can reconsider and correct the whole text at once, rather than only choosing the next word at the end.

Why it is theoretically better

At first glance the transformer approach looks more efficient: why run several iterations when you can generate in a single pass? But research shows something unexpected. The diffusion process gives models two fundamental advantages:

• Context reassessment — at each iteration the model can change decisions it made earlier
• Parallelism — the text is updated across its whole length at once, not token by token
• Fewer internal contradictions — the model sees the entire text and can logically reconcile its different parts
• A new research paradigm — which architectures are possible in NLP is an open question

In practice, diffusion models can run faster on some tasks thanks to parallelism. Instead of waiting for token N before refining positions N+1 through N+50, the system can improve all of them at the same time.

Status: lab, not production

DiffusionGemma is not a replacement for ChatGPT. It is a research model that Google released as open source for community experiments. So far its results lag behind the leading transformers on standard benchmarks.

But it is the first serious challenge to the monopoly of the transformer architecture. Google researchers note that the diffusion approach looks especially promising for tasks that demand deep logical processing: multi-step inference, complex reasoning, and creative writing with internal consistency.

What this means

Google is sending a clear signal: the paradigm that "the transformer is the only right path" is incomplete. Alternative architectures exist that may become competitive, and even superior, in specific areas. The likely direction of development is hybrid models that combine transformer speed with the diffusion approach's capacity for deep reworking of information.