Live Nov 18 · Cohort 3Nov 18, 19:00 MSK — free hands-on AI agents live workshop · Live Cohort 3 starts Nov 23Join Nov 18 Live →Cohort 3 (Nov 23)
ReleaseInsight · On RadarMicrosoft Research→ original

TokenEconomizer plugin cuts AI coding agent costs by 18.4–30.9% without changing the model

A coding agent can work in the same repository for weeks, yet it starts every session from scratch. It figures out again how to run the tests and drags along tools it does not need. The TokenEconomizer plugin reads the logs of past sessions and feeds what it has learned back to the agent. On 277 bug fixes from SWE-Bench, task cost fell by 18.4–30.9%. For teams working in long-lived repositories, that is a saving without switching agents.

AI-processed from Microsoft Research; edited by Hamidun News
TokenEconomizer plugin cuts AI coding agent costs by 18.4–30.9% without changing the model
Source: Microsoft Research. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceMicrosoft Research ↗
Editor-in-Chief
Key metric #1Of the 23 tool descriptions sent on every turn, the agent did not call 16
Key metric #2The 17% of tasks that failed consumed 32% of spending — the plugin catches looping
Key metric #3By the sixth task, cumulative costs in every repository are lower than without the plugin

If you run Claude Code or another coding agent against the same repository over and over, you have probably noticed that every new session starts as if the agent were seeing the project for the first time. It works out again how to run the tests, reads the same files again, and steps on the same rakes again. Each of those laps is paid for in tokens, which means money.

Researchers from UC Berkeley, Microsoft Research, and Microsoft Security have released TokenEconomizer, an open-source plugin that analyzes completed sessions and hands the agent what it learned in the next ones. On 277 real bug fixes, the average cost of a task fell by 18.4–30.

9%, roughly a fifth to a third of the token bill. The agent and its model stay the same, and the share of solved tasks did not drop overall. If your team has spent years working in the same few repositories, part of your bill goes to things the agent already knew yesterday.

Where the tokens go

The authors first looked at where the budget goes. They ran Claude Code on 60 tasks from SWE-Bench (a set of real bug fixes in open-source Python projects), executing each repository's tasks back to back. They found three sources of waste.

First, the agent rediscovers what is already known. Once a repository had at least five completed tasks, 30% of tool calls repeated an action from an earlier task in the same repository. Only 7% matched history from a different repository. So experience is useful locally, not in general.

Second, the agent pays for tools it does not use. On every turn it was sent 23 tool descriptions, but only 7 were ever called. The other 16 simply bloated the request.

Third, getting stuck. The 17% of tasks that failed consumed 32% of spending: the agent circles around the same error while tokens burn.

The authors also tested three popular open-source plugins for saving tokens, each with a hard-coded strategy. Each cut costs in some cases and not in others. The conclusion: there is no single recipe, and savings have to adapt to the specific repository.

How it works

After each session, TokenEconomizer reads the log and updates its knowledge of the repository. Three mechanisms then kick in, all through interfaces the agent already has.

  • Micro-skills. Short one-line notes such as "tests here are run like this." A note enters the starting context only after several sessions have confirmed the pattern. If contradicting data appears later, the note is withdrawn.
  • Tool pruning. The plugin removes tools that, judging by the repository's history, the agent is unlikely to need. The basic ones (running commands and editing files) always stay, and removed tools can be restored.
  • Intervention on looping. A separate judge model decides whether the agent is really stuck or still making progress. If it is stuck, the agent gets a short prompt to try a different approach. If nothing changes even after that, the open-source version of the plugin does not cut the run off itself but warns the developer.

The plugin does not require its own API key and stores what it learns on the developer's machine. The reported savings already include the cost of its own model calls.

What the measurements showed

The evaluation covered 277 SWE-Bench tasks from six repositories, with five agent-and-model combinations. In every repository, the plugin started with no history. Average task cost fell by 18.4–30.9%, and Pass@1 (the share of tasks solved on the first attempt) rose in four of the five combinations. Among approaches that do not sacrifice quality, the plugin saved the most in all 30 repository-combination pairs. For comparison, simply reducing the model's reasoning depth saved as much or more, but noticeably worsened results.

The savings build up gradually. The first tasks may even cost more while the plugin is learning, but by the sixth task, cumulative costs are below the baseline in every repository. Each of the three mechanisms pays off on its own, and together they deliver the maximum.

The effect held up beyond bug fixes. On Web-Bench, which involves sequential web development, costs fell by 16.0–21.2%. On security auditing, where every task runs in a new repository and there is no accumulated experience, it was still 6.4–19.6% cheaper.

What to do with this

The plugin installs alongside your usual agent, so trying it is cheap. There are caveats, though. This is a research project, and the main figures come from benchmarks of Python bug fixes, which differ from production codebases. The gain is not visible from the first task but only after history accumulates, so it will do little for one-off experiments. Before rolling the plugin out to a team, measure costs on your own repository before and after.

Story

Key Questions Answered

Where the tokens go?

The authors first looked at where the budget goes. They ran Claude Code on 60 tasks from SWE-Bench (a set of real bug fixes in open-source Python projects), executing each repository's tasks back to back. They found three sources of waste. First, the agent rediscovers what is already known. Once a repository had at least five completed tasks, 30% of tool calls repeated an action from an earlier task in the same repository. Only 7% matched history from a different repository.

How it works?

After each session, TokenEconomizer reads the log and updates its knowledge of the repository. Three mechanisms then kick in, all through interfaces the agent already has. Micro-skills. Short one-line notes such as "tests here are run like this." A note enters the starting context only after several sessions have confirmed the pattern. If contradicting data appears later, the note is withdrawn. Tool pruning. The plugin removes tools that, judging by the repository's history, the agent is unlikely to need.

What the measurements showed?

The evaluation covered 277 SWE-Bench tasks from six repositories, with five agent-and-model combinations. In every repository, the plugin started with no history. Average task cost fell by 18.4–30.9%, and Pass@1 (the share of tasks solved on the first attempt) rose in four of the five combinations. Among approaches that do not sacrifice quality, the plugin saved the most in all 30 repository-combination pairs.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Hands-On AI Engineering · Hamidun Academy

Build & Ship Production AI Agents Without Boilerplate Code

Start your free 72-hour trial at Hamidun Academy (academy.hamidun.com) — interactive lessons on Claude Code & Cursor, live Tech Radar, and production agent templates.

Start 72h Free Trial →Vibe-Coding Intensive
What do you think?
Loading comments…