Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
InfrastructureResearch Paper · For R&DCoreWeave Blog→ original

CoreWeave Trains DeepSeek-V3 671B in 17.65 Minutes on Vera Rubin, 2.7x Faster Than GB200

CoreWeave ran Nvidia's new Vera Rubin chips against the previous GB200 generation in the MLPerf Training v6.1 benchmark. DeepSeek-V3 with 671B parameters reached the target quality in 17.65 minutes on 256 GPUs across four racks, while GB200 needed 48.40 minutes with the same GPU count, which is 2.7x longer. When choosing a cloud for training, compare time-to-target on your own GPU count, not the chip's spec sheet.

AI-processed from CoreWeave Blog; edited by Hamidun News
CoreWeave Trains DeepSeek-V3 671B in 17.65 Minutes on Vera Rubin, 2.7x Faster Than GB200
Source: CoreWeave Blog. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceCoreWeave Blog ↗
Editor-in-Chief
Llama 3.1 8B on a single rack7.16 minutes versus 13.70 on GB200
Key metric #2The speedups were calculated by CoreWeave itself; MLCommons did not verify them
Qwen3.5 397B with RL22.78 minutes on 256 GPUs, 128 for training and 128 for generation

Imagine you rent a cluster to train a large model and pay for every hour. Cloud advertising usually highlights the chips' spec-sheet numbers, but those don't tell you how many hours training will actually take. That is what the MLPerf Training benchmark is for: it times how long a system takes to bring a given model to a target quality.

In the latest v6.1 round, cloud provider CoreWeave trained DeepSeek-V3 with 671B parameters in 17.65 minutes on 256 Nvidia Rubin GPUs, housed in four Vera Rubin NVL72 racks (72 GPUs per rack).

The previous generation, GB200 NVL72, took 48.40 minutes on the same GPU count at the same provider. The difference is 2.

7x, which is how much time and money one training run can save.

What Was Measured and What the Numbers Show

MLPerf Training v6.1 does not measure a chip's theoretical maximum. It measures the performance of the whole stack: accelerators, networking, job scheduling and software. CoreWeave competed in the Available category of the closed division, meaning hardware that customers can actually rent. Systems ranged from a single rack of 72 GPUs to four racks of 256 GPUs, and models ranged from 8 to 671 billion parameters.

The most notable result is DeepSeek-V3, which uses a mixture-of-experts (MoE) architecture: 17.65 minutes versus 48.40 on GB200. According to the company, it was the only cloud provider to submit results on multiple Vera Rubin racks in this round.

On a single rack, the team trained Llama 3.1 8B in 7.16 minutes. That is the fastest result among submissions on 72 GPUs in the round, and 1.9x faster than CoreWeave's own GB200 NVL72 run, which took 13.70 minutes. Both runs used NVFP4 for matrix multiplications, FP8 for attention and CUDA graphs. By the company's estimate, one rack could theoretically complete about eight such runs per hour, excluding startup and quality evaluation.

Why Multiple Racks Are Hard

In an MoE model, tokens at each layer are routed to different experts, which live on different GPUs. Think of an office where every question has to be passed to the right specialist and the answer returned. While everyone sits in one room, it is fast. Once staff are spread across four floors, everything depends on the elevators. The same applies here: inside a rack, GPUs communicate over the fast NVLink interconnect, while traffic between racks crosses the network fabric.

CoreWeave connected the racks with an Ethernet network running RoCE, using Spectrum-6 switches at 102.4 Tbit/s and SuperNIC network cards. It also uses topology-aware job placement, which tries to put communicating parts of the training job closer together. Separately, fleet health monitoring finds slow or faulty nodes before they slow down the entire run. A single lagging GPU in a large cluster can waste as much useful time as another software optimization gains.

Reinforcement Learning Fine-Tuning and Caveats

Another submission was Qwen3.5 397B with reinforcement learning (RL): 22.78 minutes on 256 GPUs across four racks. Half of the GPUs, 128 of them, handled training, and the other 128 generated responses. This benchmark is the first in MLPerf Training for post-training LLM fine-tuning, and its workload is different: the system must generate, evaluate and update the model at the same time. For this part, the company reported results on GB200 NVL72, as it did for GPT-OSS 20B.

The caveats matter. CoreWeave itself calculated the 2.7x and 1.9x speedups by comparing its Vera Rubin and GB200 submissions at the same GPU count, and MLCommons did not verify these calculations. The claim of being the "only cloud" is also based on CoreWeave's own analysis of the published results. Finally, these are reference workloads: on your dataset and your architecture, the numbers will differ.

What Practitioners Should Take Away

First, if you train large MoE models, evaluate a cloud by time-to-target on multiple racks, not by the specs of a single card. That is where networking and scheduling determine what you pay. Second, for smaller models and frequent experiments, the speed of a single run matters most. Under ten minutes per rack means more iterations per day. Third, ask providers for results at your GPU count, along with data on downtime and recovery from failures, because those are what separate a fast benchmark from fast production.

Story

Key Questions Answered

What Was Measured and What the Numbers Show?

MLPerf Training v6.1 does not measure a chip's theoretical maximum. It measures the performance of the whole stack: accelerators, networking, job scheduling and software. CoreWeave competed in the Available category of the closed division, meaning hardware that customers can actually rent. Systems ranged from a single rack of 72 GPUs to four racks of 256 GPUs, and models ranged from 8 to 671 billion parameters.

Why Multiple Racks Are Hard?

In an MoE model, tokens at each layer are routed to different experts, which live on different GPUs. Think of an office where every question has to be passed to the right specialist and the answer returned. While everyone sits in one room, it is fast. Once staff are spread across four floors, everything depends on the elevators. The same applies here: inside a rack, GPUs communicate over the fast NVLink interconnect, while traffic between racks crosses the network fabric.

What are the key details on Reinforcement Learning Fine-Tuning and Caveats?

Another submission was Qwen3.5 397B with reinforcement learning (RL): 22.78 minutes on 256 GPUs across four racks. Half of the GPUs, 128 of them, handled training, and the other 128 generated responses. This benchmark is the first in MLPerf Training for post-training LLM fine-tuning, and its workload is different: the system must generate, evaluate and update the model at the same time. For this part, the company reported results on GB200 NVL72, as it did for GPT-OSS 20B. The caveats matter.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…