CoreWeave Benchmarks Vera Rubin: Up to 10x More Tokens per Megawatt Than GB200
CoreWeave has become the first to measure NVIDIA's new Vera Rubin NVL72 rack (72 GPUs in a single cabinet) against its predecessor, the GB200 NVL72. The metric was tokens per second per megawatt of electricity. Tokens are the pieces of text a model assembles into its answer. On DeepSeek R1, at the same response speed, the gain for a single user reaches up to 10x. For agentic services, this means more requests within the same energy budget, but the figures are so far claimed only by the provider itself.
AI-processed from CoreWeave Blog; edited by Hamidun News
CoreWeave is a cloud provider that rents out GPUs for AI workloads. The company has published the first measurements of NVIDIA's new Vera Rubin NVL72 rack: a server cabinet in which 72 Rubin graphics processors work as one large accelerator. It was compared with the previous generation, the GB200 NVL72, running the reasoning model DeepSeek R1.
The metric is tokens per megawatt. A token is a piece of a word that a model assembles into its answer, and a megawatt is the electricity ceiling that can be supplied to a server hall. Put simply, the test measured how much text a data center produces per unit of energy.
At the same response speed, the new rack delivered up to 10 times more than the GB200. For those building agents, this speaks directly to the price of a token.
What Was Measured
In early June, CoreWeave said it was the first in the industry to bring up and validate the full Vera Rubin NVL72: power, cooling, networking, and compute. Now the company has shown the first measurements on real silicon. It tested DeepSeek R1, a reasoning model on which many agentic workflows are currently built.
The team did not measure raw speed alone. Throughput in tokens per second per megawatt was compared against interactivity, meaning the speed a single user or a single agent actually perceives (tokens per second per user). This is a fairer comparison than pushing maximum load: if answers arrive slowly, it does not matter to the user how many total tokens the rack produces. At comparable interactivity, the Vera Rubin NVL72 showed up to a 10x advantage in tokens per second per megawatt.
Both platforms were tested with modern optimizations enabled: large-scale expert parallelism (a mixture-of-experts model is split across many GPUs), the NVFP4 reduced-precision number format, multi-token prediction (the model predicts several tokens per step), and disaggregated prefill and decode. All of this ran on NVIDIA TensorRT-LLM and Dynamo. The comparison pits the best against the best, not new hardware against unoptimized old hardware.
Why This Matters for Agents
A reasoning model does not answer in a single pass. It plans, checks itself, and corrects itself, so it generates noticeably more tokens per request than an ordinary chatbot. Every token passes through memory and across the links between GPUs. That is why, as CoreWeave points out, responsiveness, not just raw FLOPs, determines whether an agent works instantly or lags.
The platform was built for this task: 72 Rubin GPUs and 36 Vera CPUs in one rack, joined by a fully connected NVLink 6 fabric at 260 TB/s, with native NVFP4 support built into the Transformer Engine. In practice, a 10x margin in tokens per megawatt gives two options: serve far more requests within the same power limits, or serve the same traffic at much lower power consumption. Both reduce the cost per token.
CoreWeave also cites customer expectations, without naming the companies. A global cybersecurity firm expects a 10x gain per watt in threat detection. A company with an autonomous coding agent anticipates a similar increase in requests per GPU at a substantially lower token price. An AI search engine already running on the GB200 NVL72 expects comparable throughput per GPU. Importantly, these are expectations, not measured results.
Caveats and What to Do
The numbers should be read soberly. This is a publication from the cloud provider itself, not an independent benchmark such as MLPerf. The measurement covers a single model, DeepSeek R1, and the wording is "up to 10x." The article provides no absolute tokens-per-second figures and no price per token, so the results cannot be converted into a budget. In addition, the gain was achieved with the full set of optimizations. Without Dynamo, NVFP4, and the prefill/decode split, the result would likely be more modest.
CoreWeave notes that with the GB200 NVL72, its first figures improved noticeably over several months after tuning the scheduler, networking, and software, and it expects the same trajectory here. The company promises to publish updates.
For teams building agentic services, the takeaway is simple. If your main cost is tokens from reasoning models, plan the next class of racks into your roadmap and verify in advance that your stack supports NVFP4 and disaggregated prefill/decode. Forecast price per token only after independent tests on your own workloads.
- Nov 21, 2026Open-Weight Inkling 2 Mini Matches Qwen3.8-27B on Independent Benchmark, Runs on 16 GB of Memory
- Nov 21, 2026The Programmer Stopped Being an AI Agent Dispatcher: Multica Takes Over the Handoff Routine
- Nov 21, 2026MLCommons Starts Measuring AI APIs Under Real Load: 7 Runs for Agents
- Nov 21, 2026For Three Days, a Hacked DeepSeek Answered Through OpenRouter: Refusals Fell from 83–98% to 1%
Key Questions Answered
What Was Measured?
In early June, CoreWeave said it was the first in the industry to bring up and validate the full Vera Rubin NVL72: power, cooling, networking, and compute. Now the company has shown the first measurements on real silicon. It tested DeepSeek R1, a reasoning model on which many agentic workflows are currently built. The team did not measure raw speed alone.
Why This Matters for Agents?
A reasoning model does not answer in a single pass. It plans, checks itself, and corrects itself, so it generates noticeably more tokens per request than an ordinary chatbot. Every token passes through memory and across the links between GPUs. That is why, as CoreWeave points out, responsiveness, not just raw FLOPs, determines whether an agent works instantly or lags.
What are the key details on Caveats and What to Do?
The numbers should be read soberly. This is a publication from the cloud provider itself, not an independent benchmark such as MLPerf. The measurement covers a single model, DeepSeek R1, and the wording is "up to 10x." The article provides no absolute tokens-per-second figures and no price per token, so the results cannot be converted into a budget. In addition, the gain was achieved with the full set of optimizations. Without Dynamo, NVFP4, and the prefill/decode split, the result would likely be more modest.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.