Nunchaku Lite: 4-битная генерация изображений в Diffusers — до 50% меньше VRAM

HuggingFace released NunchakuLite on July 23, 2026 — a native integration of 4-bit inference for diffusion models into the Diffusers library, which cuts video memory usage by up to 50% and speeds up image generation by roughly 30%.

What Are Nunchaku and SVDQuant

Nunchaku is a CUDA inference engine that implements the SVDQuant quantization method for diffusion transformers. The NunchakuLite integration is the first to bring this technology directly into Diffusers: the model loads with the standard `from_pretrained()`, without a separate pipeline or manual compilation, and the required CUDA kernels are downloaded automatically via the `kernels` package on first run.

The key difference between Nunchaku and most backends is the W4A4 mode: both weights and activations are compressed to 4 bits. Ordinary weight-only methods reduce memory but don't speed up computation; here, the denoising loop itself is accelerated.

"SVDQuant migrates activation outliers into the weights, represents the heaviest part of each weight matrix with a small 16-bit low-rank branch, and quantizes the remaining residual to 4 bits,"

HuggingFace's technical blog states.

The SVDQuant method is based on a paper published in November 2024 on arXiv by the MIT HAN Lab team. For heavy layers (attention and MLP), the `svdq_w4a4` kernel with low-rank correction is used, while for modulation layers, `awq_w4a16` is used with 4-bit weights and 16-bit activations.

How Much Faster Is Generation

On an RTX PRO 6000 GPU at 1024×1024 resolution, NunchakuLite delivers a measurable gain across all metrics. The baseline BF16 run takes 3,00 seconds and 31,1 GB of video memory; in NVFP4 format, it's already down to 2,27 seconds and 20,6 GB.

• BF16 (baseline): 3,00 s, 31,1 GB memory — 1,0x speedup
• NVFP4: 2,27 s, 20,6 GB — 1,35x speedup
• NVFP4 + torch.compile: 1,68 s, 20,6 GB — 1,8x speedup
• NVFP4 + NF4 text encoder: 2,29 s, 16,0 GB memory
• ERNIE-Image-Turbo model: 1024×1024 in ~1,7 s on RTX 5090 (~12 GB vs. ~24 GB in BF16)

Pairing this with `torch.compile` speeds up generation by up to 1,8x, and swapping the text encoder for NF4 drops peak memory to 16,0 GB — enough to run heavy models on consumer GPUs.

What Hardware and Models Are Supported

NunchakuLite works on most modern NVIDIA GPUs, but the quantization scheme depends on the architecture. The NVFP4 format via the `svdq_w4a4` kernel requires Blackwell (RTX 50, RTX PRO 6000, B200), while Turing, Ampere, and Ada (RTX 30/40, A100, L40S) use INT4. Volta and Hopper architectures are not supported — the validator immediately returns a clear error on load.

Ready-to-run models have already been published under the lite-infer organization on the HuggingFace Hub: ERNIE-Image-Turbo (in INT4 and NVFP4 variants), Krea 2 Turbo (NVFP4), and FLUX.2 Klein 4B. For custom quantization, the team released the diffuse-compressor toolkit — it inspects the model, runs SVDQuant calibration, assembles a Diffusers pipeline, and publishes the result to the Hub.

What This Means

4-bit inference is no longer confined to isolated forks and is now arriving in the mainstream image-generation library. Developers get half the VRAM usage and a noticeable speedup with essentially a single line of code — this makes flagship diffusion models accessible on ordinary gaming GPUs.

Frequently Asked Questions

Which GPUs does NunchakuLite work on?

The NVFP4 format requires Blackwell cards — RTX 50, RTX PRO 6000, or B200. On RTX 30/40, A100, and L40S, the INT4 scheme works. Volta and Hopper architectures aren't supported at all.

How much less video memory is needed?

On the RTX PRO 6000, peak VRAM drops from 31,1 GB in BF16 to 20,6 GB in NVFP4, and with the NF4 text encoder — to 16,0 GB, i.e., almost half.