Starts Nov 23Live Cohort 3 of Vibe Coding & AI Agents: 3 weeks, 9 live hands-on sessionsCohort 3 (Nov 23) →
ReleaseAPI / Product · ActionableThe Decoder→ original

Hugging Face routes LLM requests through OpenRouter, cutting capacity errors by 46%

Previously, a model request on Hugging Face failed if the provider ran out of free capacity. Now Inference Providers sends calls through OpenRouter, which picks a provider by price, speed and reliability and switches to another one if a provider fails. According to OpenRouter's measurements, capacity errors fell by 46%, and the first token arrives 16.9% faster. You don't need to change your code, but your own API keys for LLMs no longer work.

AI-processed from The Decoder; edited by Hamidun News
Hugging Face routes LLM requests through OpenRouter, cutting capacity errors by 46%
Source: The Decoder. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceThe Decoder ↗
Editor-in-Chief
Key metric #1":fastest" picks the fastest provider, ":cheapest" picks the cheapest one
Key metric #2Open models account for 71.7% of tokens in the OpenRouter top 50 and cost roughly four times less
Key metric #3Your own keys remain only for image and video providers

Imagine your app calls an open model through Hugging Face, and the provider you picked has just run out of free GPUs. Before, you got a "capacity shortage" error and had to find a fallback yourself: rewrite the config, swap the key, ship a fix. Now Hugging Face Inference Providers sends language model requests through OpenRouter, which picks a provider for each call based on price, speed and reliability. By OpenRouter's own measurements, capacity errors under the new setup are 46% less frequent. For developers, that means fewer night-time on-call shifts, but it comes at a price: your own API keys for language models no longer work.

How the routing works

By default, the router looks for a balance of price, speed and reliability. It works like a ride-hailing app: you give an address, and it finds a car that is closer and cheaper. If a driver cancels, another one arrives. The same logic applies here: if one provider fails, the request automatically goes to another. You don't need to change your existing code.

You can steer the choice by adding simple suffixes to the model name. The ":fastest" suffix sends the request to the provider with the highest throughput, meaning the most tokens per second. The ":cheapest" suffix picks the cheapest one. You can also specify a particular provider by name. The "X-HF-Session-Id" header helps the router take provider caches into account. That is useful in long conversations: you don't pay each time to reprocess the same context.

Models are available through OpenAI's Chat Completions and Responses APIs, as well as Anthropic's Messages API. Hugging Face has published setup guides for Claude Code, Codex, OpenCode, Pi and Hermes Agent. That means you can point popular coding agents at open models without homemade shims.

What the measurements showed and what changed in billing

According to OpenRouter, capacity errors dropped by 46% with the new routing. Time to first token, the delay before a response starts, improved by 16.9%. Throughput rose by 16.3%. The partner ran the measurements itself, and the article cites no independent verification. OpenRouter also claims 800 trillion tokens per month, more than 10 million users and a catalog of 600+ models from 80+ providers.

Payment has become simpler: Hugging Face passes provider prices through without a markup and issues the bill itself. Pro subscribers get a small monthly credit, while team and enterprise organizations get a credit per seat. Model pages now have tabs with providers and their metrics, and you can turn off providers you don't want in settings.

There is also a downside. If you signed a direct contract with a provider for discounts or special terms, you can no longer use it through Hugging Face: keys for language models don't work. They remain only for image and video providers, and generation of images, video and audio still goes directly through Inference Providers. Plans include linking an OpenRouter account to Hugging Face and moving other task types to OpenRouter.

Open models have become the main traffic

Hugging Face cites data on a market shift. The share of open models in weekly tokens among OpenRouter's 50 most popular models rose from 26.8% in October 2025 to 71.7% at the end of September 2026. In July it briefly reached 77.5%. The reason is price: on OpenRouter, open models cost on average about four times less than closed ones. Since spring 2026, they have processed more than half of the tokens among the most in-demand models.

A quality gap remains, though. The best open model, Kimi K3.1 Pro, scores 46 points on the updated Artificial Analysis intelligence index. Claude Fable 5.2 scores 65 points but costs roughly seven times more per task. For high-volume scenarios such as classification, data extraction and draft replies, open models are the better value; for complex reasoning, closed models are still ahead.

What is behind the deal

Both companies are on the verge of changing owners. In late August, it was reported that Nvidia is buying Hugging Face for $12.9 billion. The US Department of Justice is reviewing the deal over concerns that Nvidia would gain control of a key model distribution platform and restrict competitors. Closing is expected in the first half of 2027. In August, Stripe agreed to buy OpenRouter for more than $7 billion, expecting to earn money from token billing.

The practical takeaway is simple. If you build a product on open models, the new setup lowers the risk of downtime and simplifies billing. But you become dependent on a chain of two companies that will soon change owners. It makes sense to keep a fallback route directly to a provider and to compare task costs on ":cheapest" and ":fastest" using your own data.

Story

Key Questions Answered

How the routing works?

By default, the router looks for a balance of price, speed and reliability. It works like a ride-hailing app: you give an address, and it finds a car that is closer and cheaper. If a driver cancels, another one arrives. The same logic applies here: if one provider fails, the request automatically goes to another. You don't need to change your existing code. You can steer the choice by adding simple suffixes to the model name.

What the measurements showed and what changed in billing?

According to OpenRouter, capacity errors dropped by 46% with the new routing. Time to first token, the delay before a response starts, improved by 16.9%. Throughput rose by 16.3%. The partner ran the measurements itself, and the article cites no independent verification. OpenRouter also claims 800 trillion tokens per month, more than 10 million users and a catalog of 600+ models from 80+ providers. Payment has become simpler: Hugging Face passes provider prices through without a markup and issues the bill itself.

What are the key details on Open models have become the main traffic?

Hugging Face cites data on a market shift. The share of open models in weekly tokens among OpenRouter's 50 most popular models rose from 26.8% in October 2025 to 71.7% at the end of September 2026. In July it briefly reached 77.5%. The reason is price: on OpenRouter, open models cost on average about four times less than closed ones. Since spring 2026, they have processed more than half of the tokens among the most in-demand models. A quality gap remains, though.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…