Nunchaku Lite: 4-bit Image Generation in Diffusers — Up to 50% Less VRAM

On July 23, 2026, Hugging Face released Nunchaku Lite, a native integration of 4-bit diffusion model inference into the Diffusers library. It cuts VRAM usage by up to 50% and speeds up image generation by roughly 30%.

What Are Nunchaku and SVDQuant?

Nunchaku is a CUDA inference engine that implements the SVDQuant quantization method for diffusion transformers. Nunchaku Lite is the first integration to bring this technology directly into Diffusers: the model loads with a regular `from_pretrained()` call, with no separate pipeline and no manual compilation. The required CUDA kernels download automatically through the `kernels` package on first run.

The key difference between Nunchaku and most other backends is its W4A4 mode, which compresses both weights and activations to 4 bits. Ordinary weight-only methods reduce memory but do not speed up computation. Here, the denoising loop itself gets faster.

"SVDQuant moves activation outliers into the weights, represents the heaviest part of each weight matrix with a small 16-bit low-rank branch, and quantizes the remaining residual to 4 bits," the

Hugging Face technical blog says.

SVDQuant is based on a paper published on arXiv in November 2024 by the MIT HAN Lab team. For the heavy layers (attention and MLP), the `svdq_w4a4` kernel runs with low-rank correction. For modulation layers, the `awq_w4a16` kernel is used, with 4-bit weights and 16-bit activations.

How Much Faster Is Generation?

On an RTX PRO 6000 at 1024×1024 resolution, Nunchaku Lite delivers measurable gains across all metrics. The baseline BF16 run takes 3.00 seconds and 31.1 GB of VRAM; in NVFP4 format, it takes 2.27 seconds and 20.6 GB.

• BF16 (baseline): 3.00 s, 31.1 GB of memory — 1.0x speedup
• NVFP4: 2.27 s, 20.6 GB — 1.35x speedup
• NVFP4 + torch.compile: 1.68 s, 20.6 GB — 1.8x speedup
• NVFP4 + NF4 text encoder: 2.29 s, 16.0 GB of memory
• ERNIE-Image-Turbo model: 1024×1024 in ~1.7 s on an RTX 5090 (~12 GB versus ~24 GB in BF16)

Combining it with `torch.compile` speeds up generation by up to 1.8x, and swapping the text encoder for NF4 drops peak memory to 16.0 GB. That is enough to run heavy models on consumer cards.

Which Hardware and Models Are Supported?

Nunchaku Lite runs on most modern NVIDIA cards, but the quantization scheme depends on the architecture. The NVFP4 format via the `svdq_w4a4` kernel requires Blackwell (RTX 50, RTX PRO 6000, B200), while Turing, Ampere and Ada (RTX 30/40, A100, L40S) use INT4. The Volta and Hopper architectures are not supported, and the validator raises a clear error at load time.

Ready-to-run models are already published in the lite-infer organization on the Hugging Face Hub: ERNIE-Image-Turbo (in INT4 and NVFP4 variants), Krea 2 Turbo (NVFP4) and FLUX.2 Klein 4B. For custom quantization, the team released the diffuse-compressor toolkit. It inspects a model, runs SVDQuant calibration, builds a Diffusers pipeline and publishes the result to the Hub.

What This Means

4-bit inference is no longer confined to individual forks and is arriving in the main image generation library. Developers get half the VRAM usage and a noticeable speedup with effectively one line of code, which makes flagship diffusion models accessible on ordinary gaming GPUs.

Frequently Asked Questions

Which GPUs does Nunchaku Lite work on?

The NVFP4 format requires Blackwell cards: RTX 50, RTX PRO 6000 or B200. On RTX 30/40, A100 and L40S, the INT4 scheme is used. The Volta and Hopper architectures are not supported at all.

How much less VRAM is needed?

On the RTX PRO 6000, peak VRAM drops from 31.1 GB in BF16 to 20.6 GB in NVFP4, and to 16.0 GB with the NF4 text encoder, which is nearly half.