NVIDIA Nemotron-Labs-3-Puzzle-75B: 37% model compression doubled server speed

On July 9, 2026, NVIDIA released Nemotron-Labs-3-Puzzle-75B, a compressed version of the Nemotron-3-Super language model. NVIDIA engineers used the Puzzle methodology, which alternates hardware-oriented compression with knowledge recovery phases. As a result, the model shrank by 37% — from 120.7B to 75.3B parameters, but server-class performance increased by 2.03x. On a single node with eight B200 accelerators, the model serves 100 tokens per second per user. On a single H100 with a maximum context of 1 million tokens, throughput jumped from 1 to 8 concurrent requests.

How the Puzzle Method Works

Nemotron-Labs-3-Puzzle-75B was created using a new Puzzle method, which solves a classic problem: when compressing a model, it loses capabilities. NVIDIA engineers found a solution — alternating two steps.

At the structural compression step, the algorithm removes or prunes neurons based on energy consumption and throughput on specific hardware. This is not just weight pruning, but an analysis of the hardware profile. Then comes a short knowledge distillation phase — the original Nemotron-3-Super model "teaches" the compressed version, recovering lost knowledge.

The process repeats iteratively: compression → distillation → compression → distillation. The result — the model retains the ability to answer complex questions, write code, and analyze context, but requires 37% less memory.

• Original model: 120.7B parameters, of which 12.8B are active simultaneously
• Compressed model: 75.3B parameters, of which 9.3B are active
• Architecture type: hybrid Mixture of Experts (MoE)
• Method: iterative hardware compression and knowledge distillation
• Memory savings: approximately 37% reduction

Performance on Different Accelerators

The main advantage of Nemotron-Labs-3-Puzzle-75B is throughput, not individual request speed. This is critical for cloud APIs and enterprise services that simultaneously serve thousands of users.

On the latest hardware — a node with eight NVIDIA B200 accelerators — the Puzzle model shows 2.03x higher throughput than Nemotron-3-Super. Each user is guaranteed a minimum processing speed of 100 tokens per second.

On more accessible H100s, the benefit is even more impressive. With a context of 1 million tokens, parallelism grew from 1 to 8 concurrent requests. That is, one H100 now serves 8 times more users at the same latency.

Why This Matters for the Industry

The results of Nemotron-Labs-3-Puzzle-75B indicate that the era of giant, unoptimized models is ending. Companies value not the size of the model, but the quality-to-performance ratio on specific hardware. NVIDIA showed that even models with tens of billions of parameters can be rationally compressed.

The Puzzle method will likely become a standard for preparing LLMs for industrial deployment. Startups and mid-size companies will get a tool to take powerful models and optimize them for their data centers — whether cloud or edge devices.

What This Means

NVIDIA demonstrates that the future belongs to optimized, not giant models. 37% compression with 200% acceleration — this is not a compromise, but clear progress. Companies deploying LLMs will get a tool to save money on computing without losing functionality.