How GigaChat3 Works: The Diffusion-Based GFusion

Sberbank's GigaChat3 neural network appeared in diffusion mode in July 2026. The project, called GFusion, moves an autoregressive model to a fundamentally new way of working without full retraining.

How diffusion LLMs differ from the familiar ones

Autoregressive models such as GPT, LLaMA and the base GigaChat generate text token by token, strictly left to right. A token that has been written cannot be revisited: if the model "misses" early on, the only fix is to regenerate the entire answer. Diffusion language models (dLLMs) work differently. They start from a fully noised or masked sequence and iteratively "develop" the text over several passes, much as diffusion models reconstruct an image from noise.

The theoretical advantages: a dLLM sees the whole context at once, including the right-hand part of text not yet written, which could improve the coherence of long answers. Parallel denoising also opens room for faster inference. The project's authors themselves stress that the field is "relatively new" and that many ideas are only beginning to be tested in practice at real scale.

How the team did the conversion

The project's key bet is resource efficiency. Pretraining a 10-billion-parameter model from scratch demands enormous amounts of GPU time and many months of infrastructure work. The GigaChat Pretrain team took a fundamentally different route: start from a ready autoregressive checkpoint and adapt it to the diffusion paradigm, keeping everything accumulated during pretraining.

GigaChat3-10B-A1.8B-base is a sparse model: 10 billion parameters in total, but only 1.8 billion are active on each pass (an MoE-type architecture), which lowers the compute load at inference. The experiment produced two checkpoints with the GFusion suffix.

Project parameters:

• Base model: GigaChat3-10B-A1.8B-base (10B parameters, 1.8B active at inference)
• Approach: AR → dLLM conversion without pretraining from scratch
• Results: GFusion-10B-A1.8B-base and GFusion-10B-A1.8B
• Team: GigaChat Pretrain, Sberbank
• Timeline: internship project, several months

The format of the experiment is notable in itself: a non-trivial architectural transition was carried out as part of an internship, which suggests the tooling is accessible to teams that lack the resources of large labs.

Why this matters for Russian-language AI

GigaChat3 is one of the key open models optimized for the Russian language and for Sberbank's tasks. Converting it to diffusion mode preserves the model's linguistic "memory": all the knowledge, patterns and linguistic features accumulated during pretraining remain intact. Only the generation mechanism changes, not what the model knows but how it expresses it.

Training a high-quality Russian-language model from scratch is considerably harder than adapting an existing one: a shortage of quality training data and the high cost of pretraining make "reworking" ready checkpoints strategically attractive. If the GFusion approach proves real gains in quality or speed, it could be replicated on other existing Russian-language models without months of repeated pretraining.

What this means

GFusion shows that converting autoregressive LLMs to diffusion mode is a practically solvable task, achievable even within an internship. If further experiments confirm the approach's advantages, it would open a resource-efficient path to modernizing an entire class of language models, without the cost of a full pretraining cycle.

How does diffusion mode differ from a regular neural network?

Autoregressive models like GPT and the base GigaChat generate text token by token from left to right, and what has been written cannot be rewritten. Diffusion mode works on a different principle: parallel generation.

What is GFusion from Sberbank?

It is an experimental project in which Sberbank converted the GigaChat3-10B language model to diffusion mode without full retraining from scratch, creating two new checkpoints.