NVIDIA Compresses Kimi K3.1 Pro to 4 Bits: 2.8 Trillion Parameters Fit in About 1.4 TB
NVIDIA has published Kimi K3.1 Pro from Moonshot AI on Hugging Face in the NVFP4 format, which stores the weights in 4 bits instead of the usual 16. The model is huge: 2.8 trillion parameters in total, with 104 billion working on each token. It's like vacuum packing: the contents are almost the same, but the weights take about 1.4 TB instead of 5.6 TB. For self-hosting, this significantly lowers the memory requirements, but the quality should be checked on your own tasks. NVIDIA Model Optimizer handled the quantization. The post includes no quality benchmarks.
AI-processed from TG: @cgevent (Метаверсище и ИИще); edited by Hamidun News
Large open models often run into a simple problem: there is nowhere to run them. Weights several terabytes in size don't fit on an ordinary server, and only a few organizations can afford a GPU cluster for them. That is why NVIDIA's publication on Hugging Face deserves attention. The company has released Kimi K3.1 Pro from Moonshot AI in the NVFP4 format, a 4-bit packing of the weights instead of the standard 16 bits. By rough estimate, the weights of the 2.8-trillion-parameter model now take about 1.4 TB instead of 5.6 TB.
What was actually published
According to the model card, Kimi K3.1 Pro is a multimodal model with a Mixture of Experts (MoE) architecture. It has 2.8 trillion parameters in total, but only 104 billion are used to process each token. Think of a huge library where, for each question, you open only a few relevant shelves rather than flipping through every book. The model accepts text and images, and its stated context limit is 1,048,576 tokens, roughly one million.
NVIDIA prepared the build itself using its NVIDIA Model Optimizer tool. According to the description, the purpose of the format is to reduce the size of the weights so the model is easier to deploy on your own hardware.
What NVFP4 means in practice
Quantization means storing each number in fewer bits. Normally, one parameter takes 16 bits. NVFP4 is NVIDIA's 4-bit floating-point format: values are grouped into small blocks, and each block has its own scaling factor. This helps preserve more accuracy at such dense packing.
A rough calculation:
- 16 bits: 2.8 trillion parameters take about 5.6 TB for the weights alone.
- 8 bits: about 2.8 TB.
- 4 bits: about 1.4 TB.
This estimate excludes scaling factors and metadata, so actual file sizes will be slightly larger. Still, the nearly fourfold difference from 16 bits directly answers how many GPUs you need to buy or rent. One important caveat: a MoE model has fewer active parameters, but all experts still have to reside in memory, or they would have to be loaded from disk on the fly. The savings come from the size of all the weights, not from the 104 billion active ones.
What to check before running it
The original post includes no quality or speed measurements, so it is too early to judge how much accuracy the model lost after compression. Any quantization is a trade-off, and a 4-bit build of a model this size should be tested on your own workloads: code, long documents, and image tasks.
Other practical questions remain:
- Memory for context. The weights are only part of the footprint. If you actually use a million-token context, you need additional memory for the key-value cache, and quantizing the weights does not reduce that.
- Hardware. NVFP4 was designed with NVIDIA's newer-generation GPUs in mind. On older cards, the memory savings may not translate into a speed gain. Check the documentation.
- Serving tools. Check the model card for which inference engines support this build and which versions they require.
- License. The terms of use are listed in the model card. For commercial use, read them before deployment.
Why it matters
For teams building agents and wanting to keep a model in-house, because of data requirements or API costs, the release points to a trend: hardware vendors are now preparing compressed versions of large open models themselves. This lowers the barrier to entry but does not eliminate the work. First run the build against your own tests and compare it with a lighter alternative, then calculate the cost of ownership. If the quality holds up on your tasks, a 4-bit version of a model at this scale may turn out to be substantially cheaper to operate than the full-precision one.
- Nov 21, 2026Meta* Releases XRStereo: Nearly 18,000 Stereo Frame Sets for 3D Vision
- Nov 21, 2026Qdrant runs embeddings on an ESP32 microcontroller: a phrase becomes a vector in 1.2 ms
- Nov 21, 2026Alibaba Open-Sources AI Code Reviewer That Tens of Thousands of Developers Used for Two Years
- Nov 21, 2026Sandflow Releases 106,000 3D Models With Editable Python Code
Key Questions Answered
What was actually published?
According to the model card, Kimi K3.1 Pro is a multimodal model with a Mixture of Experts (MoE) architecture. It has 2.8 trillion parameters in total, but only 104 billion are used to process each token. Think of a huge library where, for each question, you open only a few relevant shelves rather than flipping through every book. The model accepts text and images, and its stated context limit is 1,048,576 tokens, roughly one million. NVIDIA prepared the build itself using its NVIDIA Model Optimizer tool.
What NVFP4 means in practice?
Quantization means storing each number in fewer bits. Normally, one parameter takes 16 bits. NVFP4 is NVIDIA's 4-bit floating-point format: values are grouped into small blocks, and each block has its own scaling factor. This helps preserve more accuracy at such dense packing. A rough calculation: 16 bits: 2.8 trillion parameters take about 5.6 TB for the weights alone. 8 bits: about 2.8 TB. 4 bits: about 1.4 TB. This estimate excludes scaling factors and metadata, so actual file sizes will be slightly larger.
What to check before running it?
The original post includes no quality or speed measurements, so it is too early to judge how much accuracy the model lost after compression. Any quantization is a trade-off, and a 4-bit build of a model this size should be tested on your own workloads: code, long documents, and image tasks. Other practical questions remain: Memory for context. The weights are only part of the footprint.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.