Groq Is First to Deploy NVIDIA Groq 3 LPX: 3,400 Tokens/s on Gemma 4 31B

Groq will be one of the first operators of NVIDIA Groq 3 LPX, an accelerator that boosts token generation on Vera Rubin NVL72 racks. The company will deploy both platforms in its own inference cloud, GroqCloud, with Dell Technologies handling delivery and integration. The headline figure from NVIDIA's published tests: 3400 output tokens per second on the open agentic model Gemma 4 31B at a context of 100 thousand tokens. The measurement was made in the Artificial Analysis benchmark.

What NVIDIA claims

LPX plugs into Vera Rubin NVL72 as a separate decoding layer. It raises token output speed where the context is long and an agent takes many sequential steps. The key claimed figures:

• 3400 output tokens per second on Gemma 4 31B at a 100K-token context;
• 4 times higher interactivity for latency-sensitive agentic workloads than the nearest alternative platform;
• agentic tasks such as coding finish in minutes instead of hours.

The product name shows that the LPU architecture Groq spent years developing is now coming out under the NVIDIA brand. NVIDIA's Dion Harris lays out the logic this way: agentic AI needs more tokens, fast response times, and low cost at scale. Vera Rubin handles throughput, while Groq 3 LPX handles low latency.

Why Groq is first

The company's argument: no other team has experience running LPUs in production at this scale. By its own figures, GroqCloud serves more than six million developers, Fortune 500 enterprises, and thousands of AI-native companies. The platform generates trillions of tokens per week from data centers in North America, Europe, the Middle East, and Asia-Pacific.

In August, Groq received NVIDIA Cloud Partner status. That means it designs, deploys, and operates compute clusters according to NVIDIA's reference architecture and operating standards. The new launch continues that line. According to Dell vice president Arunkumar Narayanan, Dell covers the integrated infrastructure and global supply chains. Groq CTO Sinclair Schuller calls the platform a new class of interactive inference accelerators.

What the announcement leaves out

The press release does not give per-token prices, a timeline for bringing capacity online, the number of racks deployed, or the regions where LPX will appear first. It also does not name the "nearest alternative platform" against which the 4x gain is measured. The 3400 tokens per second figure was obtained on a model with 31 billion parameters. Results for flagship closed models may differ. It is also vendor data, not independent verification on real-world workloads.

Practical takeaways for teams

If you are building agents, measure latency across the whole loop, not for a single response. A coding agent makes dozens of calls in a row, and decoding speed at long context adds up in the total task time. Until public prices and timelines exist, it makes sense to:

• measure, on your own scenarios, what share of time goes to token generation itself rather than tool calls and waiting;
• run tasks at 50–100K tokens of context, where, according to NVIDIA, the gain is largest;
• ask Groq for early testing access and token pricing terms;
• avoid tying your architecture to a single provider: open models like Gemma 4 let you compare clouds on the same workload.