MLPerf Training: 10,240-GPU clusters and 49% faster training in six months
MLCommons has published MLPerf Training v6.1, an exam for clusters: everyone trains the same model to the same quality target, and the clock decides. The largest runs grew from 8 192 to 10 240 GPUs. On the four unchanged benchmarks, average training time fell by a factor of 1.49 in six months: from 100 minutes to about 67. Part of that gain comes from software, so when choosing a cluster, check recent results rather than hardware spec sheets.
AI-processed from MLCommons; edited by Hamidun News
Imagine you are choosing where to train a model: rent GPUs in the cloud or build your own cluster. Vendors' spec-sheet numbers tell you little. MLPerf Training is a common exam: every participant solves the same task on the same data to the same target quality, and the measure is time to result.
In the latest round, v6.1 from MLCommons, the largest runs grew to 10 240 GPUs (the previous ceiling was 8 192). On the four benchmarks that did not change since the last round, the best time fell by a factor of 1.
49 on average, or roughly 49% in speed. In rough terms, training that took 100 minutes six months ago now finishes in 67. For you, this is a guide to how much real speedup you can expect from changing hardware and software settings.
What changed in this round
MLCommons received 250 submissions from 25 participants; v6.0 had 229 from 24. The largest runs used 10 240 GPUs: NVIDIA on GB300 and Azure on GB200 trained DeepSeek-V3. Google ran the same model on 8 192 TPU7x (Ironwood) chips in a single slice with a fast internal ICI network. Ironwood and NVIDIA Rubin debuted in the Available category, meaning the systems are already available to customers. In total, the round featured 12 accelerators from three vendors and four interconnect technologies between nodes.
The typical system got bigger. The median number of accelerators in a Closed result stayed at 8 from v4.0 through v6.0; now it is 32. Results on 64 or more accelerators rose to 111 from 73. Multi-node systems account for 76% of Closed results (117 of 153), versus 61% in v6.0. About 28% of submissions came from the cloud, so virtualized infrastructure is keeping pace with the rest. The University of Florida ran the benchmark on half of a B200 SuperPOD while the other half kept serving researchers.
Two benchmarks were added. The first is agentic reinforcement learning: a 397-billion-parameter model (Qwen3.5-397B) is trained to fix real software projects; seven participants ran it, with 29 submissions. The second is the generative recommender DLRMv4 (HSTU), which replaced the old DLRM-DCNv2: eight participants, 20 submissions. The large dense pretraining model was dropped after three rounds: the industry has moved to mixture-of-experts architectures, where only part of the model works on each token. DeepSeek-V3 and Qwen3.5-397B now cover that niche.
Where the 49% comes from, and why the gains are uneven
1.49 is the geometric mean of the improvement in best results across the four unchanged benchmarks. It is not an official MLPerf result but an index that MLCommons calculates itself. The systems in the comparison are the same size: 512 accelerators, except for Llama 3.1 8B, where v6.0 used 1 024. The gain is unevenly distributed: FLUX.1 sped up noticeably over six months, while the small Llama 3.1 8B barely changed. Part of the gain comes from software: Firmus and KRAI showed notable improvements over baseline implementations, including through AI-assisted optimizations.
Why more GPUs does not mean faster
The benchmark measures training time on a fixed task, so a larger cluster should finish the same work sooner. But gradient synchronization between accelerators does not shrink as the cluster grows, so 100 accelerators will not train a model 100 times faster. Batch size is also limited: Llama 3.1 8B processes about 64 sequences per step, so on hundreds of accelerators each sequence has to be split, adding data exchange inside every layer.
DeepSeek-V3 has a batch of about 16 000 sequences, but its problem is different: memory. The training state of a 671-billion-parameter model takes more than 10 TB, while an accelerator's HBM is 186–288 GB. So the model is split across pipeline stages and experts, and some computations are redone to save memory. On NVL72 racks and TPU7x slices, the fast internal network spans many nodes, so how a job divides traffic between the fast and slow networks increasingly determines speed.
Processors are a separate matter. In agentic RL (for example, Qwen3.5 GRPO), CPUs run the environments and tests that turn the model's answers into a reward. Participants used from 500 to about 3 150 parallel sandboxes. Host CPU capacity will matter more as agentic tasks grow.
What to do with this
When choosing a platform, look not at peak GPU count but at results for a task similar to yours, and at the network design. For small models, hundreds of extra accelerators will barely help. For agentic pipelines, budget CPU capacity for sandboxes.
Caveats: participants submit the results themselves, the first results on new benchmarks partly reflect their novelty, and in this round only Korea's TTA measured energy.
- Nov 19, 2026Amazon and Google will hold 28.4% of Anthropic after its IPO, but without voting rights
- Nov 19, 2026AI token prices fell 21.2% in six weeks, while corporate API spending dropped 6.3%
- Nov 19, 2026Gemini API Can Now Load Tool Schemas on Demand Instead of in Every Prompt
- Nov 19, 2026OpenCode Gets a Mobile App: A Remote Control for the Agent on Your Computer
Key Questions Answered
What changed in this round?
MLCommons received 250 submissions from 25 participants; v6.0 had 229 from 24. The largest runs used 10 240 GPUs: NVIDIA on GB300 and Azure on GB200 trained DeepSeek-V3. Google ran the same model on 8 192 TPU7x (Ironwood) chips in a single slice with a fast internal ICI network. Ironwood and NVIDIA Rubin debuted in the Available category, meaning the systems are already available to customers. In total, the round featured 12 accelerators from three vendors and four interconnect technologies between nodes.
Where the 49% comes from, and why the gains are uneven?
1.49 is the geometric mean of the improvement in best results across the four unchanged benchmarks. It is not an official MLPerf result but an index that MLCommons calculates itself. The systems in the comparison are the same size: 512 accelerators, except for Llama 3.1 8B, where v6.0 used 1 024. The gain is unevenly distributed: FLUX.1 sped up noticeably over six months, while the small Llama 3.1 8B barely changed.
Why more GPUs does not mean faster?
The benchmark measures training time on a fixed task, so a larger cluster should finish the same work sooner. But gradient synchronization between accelerators does not shrink as the cluster grows, so 100 accelerators will not train a model 100 times faster. Batch size is also limited: Llama 3.1 8B processes about 64 sequences per step, so on hundreds of accelerators each sequence has to be split, adding data exchange inside every layer.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.