CoreWeave Trains DeepSeek-V3 671B in 17.65 Minutes on Vera Rubin, 2.7x Faster Than GB200
CoreWeave ran Nvidia's new Vera Rubin chips against the previous GB200 generation in the MLPerf Training v6.1 benchmark. DeepSeek-V3 with 671B parameters reached the target quality in 17.65 minutes on 256 GPUs across four racks, while GB200 needed 48.40 minutes with the same GPU count, which is 2.7x longer. When choosing a cloud for training, compare time-to-target on your own GPU count, not the chip's spec sheet.
AI-processed from CoreWeave Blog; edited by Hamidun News
Imagine you rent a cluster to train a large model and pay for every hour. Cloud advertising usually highlights the chips' spec-sheet numbers, but those don't tell you how many hours training will actually take. That is what the MLPerf Training benchmark is for: it times how long a system takes to bring a given model to a target quality.
In the latest v6.1 round, cloud provider CoreWeave trained DeepSeek-V3 with 671B parameters in 17.65 minutes on 256 Nvidia Rubin GPUs, housed in four Vera Rubin NVL72 racks (72 GPUs per rack).
The previous generation, GB200 NVL72, took 48.40 minutes on the same GPU count at the same provider. The difference is 2.
7x, which is how much time and money one training run can save.
What Was Measured and What the Numbers Show
MLPerf Training v6.1 does not measure a chip's theoretical maximum. It measures the performance of the whole stack: accelerators, networking, job scheduling and software. CoreWeave competed in the Available category of the closed division, meaning hardware that customers can actually rent. Systems ranged from a single rack of 72 GPUs to four racks of 256 GPUs, and models ranged from 8 to 671 billion parameters.
The most notable result is DeepSeek-V3, which uses a mixture-of-experts (MoE) architecture: 17.65 minutes versus 48.40 on GB200. According to the company, it was the only cloud provider to submit results on multiple Vera Rubin racks in this round.
On a single rack, the team trained Llama 3.1 8B in 7.16 minutes. That is the fastest result among submissions on 72 GPUs in the round, and 1.9x faster than CoreWeave's own GB200 NVL72 run, which took 13.70 minutes. Both runs used NVFP4 for matrix multiplications, FP8 for attention and CUDA graphs. By the company's estimate, one rack could theoretically complete about eight such runs per hour, excluding startup and quality evaluation.
Why Multiple Racks Are Hard
In an MoE model, tokens at each layer are routed to different experts, which live on different GPUs. Think of an office where every question has to be passed to the right specialist and the answer returned. While everyone sits in one room, it is fast. Once staff are spread across four floors, everything depends on the elevators. The same applies here: inside a rack, GPUs communicate over the fast NVLink interconnect, while traffic between racks crosses the network fabric.
CoreWeave connected the racks with an Ethernet network running RoCE, using Spectrum-6 switches at 102.4 Tbit/s and SuperNIC network cards. It also uses topology-aware job placement, which tries to put communicating parts of the training job closer together. Separately, fleet health monitoring finds slow or faulty nodes before they slow down the entire run. A single lagging GPU in a large cluster can waste as much useful time as another software optimization gains.
Reinforcement Learning Fine-Tuning and Caveats
Another submission was Qwen3.5 397B with reinforcement learning (RL): 22.78 minutes on 256 GPUs across four racks. Half of the GPUs, 128 of them, handled training, and the other 128 generated responses. This benchmark is the first in MLPerf Training for post-training LLM fine-tuning, and its workload is different: the system must generate, evaluate and update the model at the same time. For this part, the company reported results on GB200 NVL72, as it did for GPT-OSS 20B.
The caveats matter. CoreWeave itself calculated the 2.7x and 1.9x speedups by comparing its Vera Rubin and GB200 submissions at the same GPU count, and MLCommons did not verify these calculations. The claim of being the "only cloud" is also based on CoreWeave's own analysis of the published results. Finally, these are reference workloads: on your dataset and your architecture, the numbers will differ.
What Practitioners Should Take Away
First, if you train large MoE models, evaluate a cloud by time-to-target on multiple racks, not by the specs of a single card. That is where networking and scheduling determine what you pay. Second, for smaller models and frequent experiments, the speed of a single run matters most. Under ten minutes per rack means more iterations per day. Third, ask providers for results at your GPU count, along with data on downtime and recovery from failures, because those are what separate a fast benchmark from fast production.
- Nov 21, 2026MLCommons Starts Measuring AI APIs Under Real Load: 7 Runs for Agents
- Nov 21, 2026For Three Days, a Hacked DeepSeek Answered Through OpenRouter: Refusals Fell from 83–98% to 1%
- Nov 21, 2026Alisa, ChatGPT and Perplexity Cite Only 34 of 309 Article Platforms
- Nov 21, 2026CoreWeave Benchmarks Vera Rubin: Up to 10x More Tokens per Megawatt Than GB200
Key Questions Answered
What Was Measured and What the Numbers Show?
MLPerf Training v6.1 does not measure a chip's theoretical maximum. It measures the performance of the whole stack: accelerators, networking, job scheduling and software. CoreWeave competed in the Available category of the closed division, meaning hardware that customers can actually rent. Systems ranged from a single rack of 72 GPUs to four racks of 256 GPUs, and models ranged from 8 to 671 billion parameters.
Why Multiple Racks Are Hard?
In an MoE model, tokens at each layer are routed to different experts, which live on different GPUs. Think of an office where every question has to be passed to the right specialist and the answer returned. While everyone sits in one room, it is fast. Once staff are spread across four floors, everything depends on the elevators. The same applies here: inside a rack, GPUs communicate over the fast NVLink interconnect, while traffic between racks crosses the network fabric.
What are the key details on Reinforcement Learning Fine-Tuning and Caveats?
Another submission was Qwen3.5 397B with reinforcement learning (RL): 22.78 minutes on 256 GPUs across four racks. Half of the GPUs, 128 of them, handled training, and the other 128 generated responses. This benchmark is the first in MLPerf Training for post-training LLM fine-tuning, and its workload is different: the system must generate, evaluate and update the model at the same time. For this part, the company reported results on GB200 NVL72, as it did for GPT-OSS 20B. The caveats matter.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.