Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
InfrastructureResearch Paper · For R&DCoreWeave Blog→ original

CoreWeave Benchmarks Vera Rubin: Up to 10x More Tokens per Megawatt Than GB200

CoreWeave has become the first to measure NVIDIA's new Vera Rubin NVL72 rack (72 GPUs in a single cabinet) against its predecessor, the GB200 NVL72. The metric was tokens per second per megawatt of electricity. Tokens are the pieces of text a model assembles into its answer. On DeepSeek R1, at the same response speed, the gain for a single user reaches up to 10x. For agentic services, this means more requests within the same energy budget, but the figures are so far claimed only by the provider itself.

AI-processed from CoreWeave Blog; edited by Hamidun News
CoreWeave Benchmarks Vera Rubin: Up to 10x More Tokens per Megawatt Than GB200
Source: CoreWeave Blog. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceCoreWeave Blog ↗
Editor-in-Chief
Key metric #1The rack holds 72 Rubin GPUs and 36 Vera CPUs, linked by NVLink 6 at 260 TB/s
Key metric #2Both racks were tested with NVFP4, multi-token prediction, and Dynamo
Key metric #3Customers are still only expecting gains; they have no measurements of their own yet

CoreWeave is a cloud provider that rents out GPUs for AI workloads. The company has published the first measurements of NVIDIA's new Vera Rubin NVL72 rack: a server cabinet in which 72 Rubin graphics processors work as one large accelerator. It was compared with the previous generation, the GB200 NVL72, running the reasoning model DeepSeek R1.

The metric is tokens per megawatt. A token is a piece of a word that a model assembles into its answer, and a megawatt is the electricity ceiling that can be supplied to a server hall. Put simply, the test measured how much text a data center produces per unit of energy.

At the same response speed, the new rack delivered up to 10 times more than the GB200. For those building agents, this speaks directly to the price of a token.

What Was Measured

In early June, CoreWeave said it was the first in the industry to bring up and validate the full Vera Rubin NVL72: power, cooling, networking, and compute. Now the company has shown the first measurements on real silicon. It tested DeepSeek R1, a reasoning model on which many agentic workflows are currently built.

The team did not measure raw speed alone. Throughput in tokens per second per megawatt was compared against interactivity, meaning the speed a single user or a single agent actually perceives (tokens per second per user). This is a fairer comparison than pushing maximum load: if answers arrive slowly, it does not matter to the user how many total tokens the rack produces. At comparable interactivity, the Vera Rubin NVL72 showed up to a 10x advantage in tokens per second per megawatt.

Both platforms were tested with modern optimizations enabled: large-scale expert parallelism (a mixture-of-experts model is split across many GPUs), the NVFP4 reduced-precision number format, multi-token prediction (the model predicts several tokens per step), and disaggregated prefill and decode. All of this ran on NVIDIA TensorRT-LLM and Dynamo. The comparison pits the best against the best, not new hardware against unoptimized old hardware.

Why This Matters for Agents

A reasoning model does not answer in a single pass. It plans, checks itself, and corrects itself, so it generates noticeably more tokens per request than an ordinary chatbot. Every token passes through memory and across the links between GPUs. That is why, as CoreWeave points out, responsiveness, not just raw FLOPs, determines whether an agent works instantly or lags.

The platform was built for this task: 72 Rubin GPUs and 36 Vera CPUs in one rack, joined by a fully connected NVLink 6 fabric at 260 TB/s, with native NVFP4 support built into the Transformer Engine. In practice, a 10x margin in tokens per megawatt gives two options: serve far more requests within the same power limits, or serve the same traffic at much lower power consumption. Both reduce the cost per token.

CoreWeave also cites customer expectations, without naming the companies. A global cybersecurity firm expects a 10x gain per watt in threat detection. A company with an autonomous coding agent anticipates a similar increase in requests per GPU at a substantially lower token price. An AI search engine already running on the GB200 NVL72 expects comparable throughput per GPU. Importantly, these are expectations, not measured results.

Caveats and What to Do

The numbers should be read soberly. This is a publication from the cloud provider itself, not an independent benchmark such as MLPerf. The measurement covers a single model, DeepSeek R1, and the wording is "up to 10x." The article provides no absolute tokens-per-second figures and no price per token, so the results cannot be converted into a budget. In addition, the gain was achieved with the full set of optimizations. Without Dynamo, NVFP4, and the prefill/decode split, the result would likely be more modest.

CoreWeave notes that with the GB200 NVL72, its first figures improved noticeably over several months after tuning the scheduler, networking, and software, and it expects the same trajectory here. The company promises to publish updates.

For teams building agentic services, the takeaway is simple. If your main cost is tokens from reasoning models, plan the next class of racks into your roadmap and verify in advance that your stack supports NVFP4 and disaggregated prefill/decode. Forecast price per token only after independent tests on your own workloads.

Story

Key Questions Answered

What Was Measured?

In early June, CoreWeave said it was the first in the industry to bring up and validate the full Vera Rubin NVL72: power, cooling, networking, and compute. Now the company has shown the first measurements on real silicon. It tested DeepSeek R1, a reasoning model on which many agentic workflows are currently built. The team did not measure raw speed alone.

Why This Matters for Agents?

A reasoning model does not answer in a single pass. It plans, checks itself, and corrects itself, so it generates noticeably more tokens per request than an ordinary chatbot. Every token passes through memory and across the links between GPUs. That is why, as CoreWeave points out, responsiveness, not just raw FLOPs, determines whether an agent works instantly or lags.

What are the key details on Caveats and What to Do?

The numbers should be read soberly. This is a publication from the cloud provider itself, not an independent benchmark such as MLPerf. The measurement covers a single model, DeepSeek R1, and the wording is "up to 10x." The article provides no absolute tokens-per-second figures and no price per token, so the results cannot be converted into a budget. In addition, the gain was achieved with the full set of optimizations. Without Dynamo, NVFP4, and the prefill/decode split, the result would likely be more modest.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…