Cloudflare Will Show Where Teams Overpay for Oversized AI Models
AI bills keep climbing, but token counts don't show where the money goes. Cloudflare has added an Overkill section to AI Gateway. It finds conversations where a simple task, such as formatting or a short summary, was sent to a heavy reasoning model. Alongside it, you can see the users and agents involved, along with cost, latency, and the number of turns. That makes it easier for teams to decide where to change their default model.
AI-processed from Cloudflare AI Blog; edited by Hamidun News
A few weeks after a company rolls out AI, the same story almost always repeats: costs rise, some requests respond more slowly, and nobody can pinpoint the cause. Developers may have taken on more complex code. Agents may have started making unnecessary clarifying calls. Or a small group of people may be spending far more than their share. Token and request counters can't tell these cases apart. Cloudflare set out to close that gap with an update to User Insights in its AI Gateway.
What the Overkill section actually shows
User Insights appeared in AI Gateway a month earlier. It showed which users, applications, and models generated traffic, and it flagged cost anomalies. Now it adds context: what kind of work sits behind a request. Is it code review, research, or an agent that made several calls in a row?
The Overkill section looks for conversations where the chosen model seems stronger than the task requires. A typical example is text formatting or a short summary sent to an expensive reasoning model. A everyday analogy fits: moving one carton of milk in a truck. It works, but it's expensive.
Cloudflare stresses that this is not a model ranking and not an automatic recommendation to switch. The section helps teams ask questions. Does the model suit the task? Does the extra capability improve the result? Could a fast, cheap model handle it? Is the problem confined to one process, one user, or one agent? The causes vary: a model is set as the default, people don't know which one to choose, or an agent is configured to use the same model for every step.
Alongside it, the Potential Savings view works. It shows requests that could potentially move to a faster or cheaper model without losing quality. You can also compare latency, input and output tokens, turn counts, and total cost for a single task type.
How the service understands the task type
The analysis groups conversations into categories: programming, research, writing, summarization, and data analysis. Turn count is analyzed separately. A long conversation isn't necessarily bad, especially in complex work. But if a simple task keeps needing several passes, it's worth checking the prompt, the model, or the process. Moreover, the cost of a task isn't determined by the first request alone. It's the sum of time, tokens, and money spent until the final result.
Technically, a separate Cloudflare Worker performs the classification. It reads the conversation trajectory: user requests, assistant replies, tool calls, and their results. The output is a category, a confidence score, and several parameters: task complexity, intent ambiguity, cost of error, and context dependence.
An important safety note: metadata is stored separately from log bodies (Durable Objects for metadata, R2 for bodies). The dashboard shows derived categories and aggregates, not the prompts themselves. The retention period for raw logs is determined by current AI Gateway settings, so check them in advance.
Another limitation: classification is asynchronous. It runs after a request is processed and adds no latency for the user. But dashboard data may lag by roughly a day. This is a tool for analyzing trends, not for monitoring traffic in real time.
Who ends up in the report, and where Auto Router fits
To link spending to people, AI Gateway uses identification. For your own applications, requests need stable user_id and session_id values, plus optionally a group and an application name in the cf-aig-metadata header. Cloudflare advises passing non-personal identifiers and not putting data about people into the prompt. For tools like Claude Code, Codex, and OpenCode, it's enough to place AI Gateway behind Cloudflare Access: user and session context is pulled in automatically. Access is free for teams of up to 50 people.
The second part of the release is Auto Router. It takes the same signals (conversation trajectory, category, complexity, and how well the model fits the task) and selects a model with cost in mind. The idea is to avoid writing a separate routing rule for each scenario. Complex code or research can still go to a powerful model, while simple tasks go to a fast, cheap one. The announcement describes Auto Router as being in both public and closed beta; sign-up runs through a separate Cloudflare post, so check access separately.
What to do in practice
If your agents and staff all use one top-tier model, start by checking what share of traffic goes to summaries, formatting, and classification, and how many turns those tasks take. If most conversations are resolved in a single turn, that's a clear candidate for a cheaper model. Cloudflare doesn't publish savings figures, so you'll need to test the hypothesis on your own data. But the approach itself is sound: understand the work first, then choose the model. All the described User Insights features are free for AI Gateway users.
- Nov 21, 2026Калифорния судится с OpenAI: её ИИ-агенты как минимум 80 раз вышли из-под контроля
- Nov 21, 2026Dreamteam Raises $53M for a CRM That Fills Itself from Email and Calls
- Nov 21, 2026Нейросеть Claude Opus 5.5 достраивает хоррор Dimension 9 из 15 комнат автора
- Nov 21, 202622 Claude Agents Spent Six Weeks on Black Hole Problems — and Solved Two
Key Questions Answered
What the Overkill section actually shows?
User Insights appeared in AI Gateway a month earlier. It showed which users, applications, and models generated traffic, and it flagged cost anomalies. Now it adds context: what kind of work sits behind a request. Is it code review, research, or an agent that made several calls in a row? The Overkill section looks for conversations where the chosen model seems stronger than the task requires. A typical example is text formatting or a short summary sent to an expensive reasoning model.
How the service understands the task type?
The analysis groups conversations into categories: programming, research, writing, summarization, and data analysis. Turn count is analyzed separately. A long conversation isn't necessarily bad, especially in complex work. But if a simple task keeps needing several passes, it's worth checking the prompt, the model, or the process. Moreover, the cost of a task isn't determined by the first request alone. It's the sum of time, tokens, and money spent until the final result.
Who ends up in the report, and where Auto Router fits?
To link spending to people, AI Gateway uses identification. For your own applications, requests need stable user_id and session_id values, plus optionally a group and an application name in the cf-aig-metadata header. Cloudflare advises passing non-personal identifiers and not putting data about people into the prompt. For tools like Claude Code, Codex, and OpenCode, it's enough to place AI Gateway behind Cloudflare Access: user and session context is pulled in automatically.
Ship AI Faster with Founders, Engineers & Weekly Architecture Reviews
Inside Vibe Club (hamidun.com/club) — weekly live masterminds with Zhemal Khamidun, Tech Radar, full LMS access, and 370+ production Claude Code skills.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.