Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseAPI / Product · ActionableNeurohive→ original

Gemini API Cuts Token Use for a 200-Tool Agent Nearly 7x, by Google's Estimate

An agent connected to MCP servers sends the model descriptions of all its tools with every request, even when it needs only one function. Google has added Tool Retrieval to the Gemini API: the model sees each tool's name and short description, and loads the parameter schema on demand. By Google's estimate, a conversation with 200 tools takes 23 650 input tokens instead of 164 000, almost 7 times fewer. This is a calculation, not a measurement, and the mode does not pay off for small tool sets.

AI-processed from Neurohive; edited by Hamidun News
Gemini API Cuts Token Use for a 200-Tool Agent Nearly 7x, by Google's Estimate
Source: Neurohive. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceNeurohive ↗
Editor-in-Chief
Key metric #1About 26 tokens per tool go to the name and description of a deferred function
Key metric #2An input request with 200 tools costs $0.035 instead of $0.246 at $1.50 per 1M tokens
Key metric #3With 10 tools and a 30 000-token context, usage comes out 36.5% higher

You connect a couple of MCP servers to an agent, add a dozen of your own functions, and the token bill climbs even though users ask the same questions. The reason: descriptions of all tools, along with their parameter schemas, go to the model with every request, even when a conversation needs only one function out of two hundred. Google addresses this in the Gemini API with Tool Retrieval.

The model keeps the full list at hand but receives a function's complete description only when it needs it. By Google's estimate, a conversation with a library of 200 tools takes about 23 650 input tokens instead of 164 000. That is almost 7 times fewer, or roughly 86%.

One caveat: this is a worked example, not a measurement. The feature is currently available as a preview in the Interactions API.

How it works

A developer marks rarely needed functions with a special deferred-loading flag. The model still sees the name and short description of each such function. According to Google's first measurements, that comes to about 26 tokens per tool. The parameter schema, the heaviest part, is not sent.

When the model needs a function, it requests it by name. The API loads the schema, and the model makes the call, all within a single request. Think of a library with a card catalog: the reader is not handed every book, but browses the cards and asks for the one they need.

The same flag can be set on an entire remote MCP server, in which case all of its tools load on demand. Loaded schemas are appended as the conversation goes on rather than placed at the start of the prompt. The start of the prompt does not change, so it keeps being served from the cache.

There is also a search mode that runs on the application side. It is meant for companies that have their own tool registry or check user permissions. Each search there adds an extra round trip, but tools do not have to be declared in advance: they can be added mid-conversation, for example when a user connects a new integration.

What the numbers say, and how rivals compare

By our calculation, at $1.50 per million input tokens, a conversation with 200 tools costs $0.246 with standard loading and about $0.035 with Tool Retrieval. Over 10 000 conversations, that saves roughly $2100. The calculation does not account for output tokens or the cache. Cached input costs $0.075 per million, which is 20 times cheaper than regular input.

Competitors claim different figures. Anthropic cited an 85% reduction in token use for its Tool search tool. OpenAI reported 47% at the same accuracy on the MCP Atlas benchmark. The companies measured under different conditions, so these percentages cannot be compared directly.

The Gemini interface nearly mirrors OpenAI's: it uses the same tool type. In our assessment, this will make it easier to move agents between platforms. Anthropic shipped its solution almost a year before Google, and OpenAI added tool search to the Responses API together with GPT-5.4. With Anthropic, the model first sees only the search tool itself, searches by regular expression or BM25, and by default gets up to 5 results. A single request can have up to 10 000 tools. With Gemini, the model sees the entire catalog of names and descriptions up front. For 1000 tools, that is about 26 000 tokens in every request (our estimate).

When it doesn't pay off

There is no separate fee for the feature; tokens are billed at standard rates. But each server-side search means another model pass over the full context. That is almost like paying for the prompt a second time, minus the cached portion.

Google itself gives a counterexample: 10 tools and a 30 000-token context. With deferred loading, the total comes to 92 830 tokens instead of 68 000, which is 36.5% more. Google advises keeping standard loading for libraries of up to about 20 stable tools. Anthropic recommends turning on search from as few as 10.

Preview limitations:

  • Tool Retrieval does not work in the classic generateContent method;
  • the interface may change before general release;
  • a loaded schema cannot be unloaded until the conversation ends.

Google publishes no measurements of tool-selection accuracy, even though it cites a drop in accuracy as one reason for the launch. Anthropic does have such data: according to its internal measurements, Claude Opus 4.5's accuracy on MCP tasks rose from 79.5% to 88.1%. These are the company's own figures, not independent verification.

The practical takeaway is simple. If your agent has dozens of tools and several MCP servers, and a conversation needs only a small fraction of them, test deferred loading on your own logs: count tokens before and after, and compare function-selection accuracy. If the tool set is small and stable, standard loading with caching is most likely cheaper and simpler.

Story

Key Questions Answered

How it works?

A developer marks rarely needed functions with a special deferred-loading flag. The model still sees the name and short description of each such function. According to Google's first measurements, that comes to about 26 tokens per tool. The parameter schema, the heaviest part, is not sent. When the model needs a function, it requests it by name. The API loads the schema, and the model makes the call, all within a single request.

What the numbers say, and how rivals compare?

By our calculation, at $1.50 per million input tokens, a conversation with 200 tools costs $0.246 with standard loading and about $0.035 with Tool Retrieval. Over 10 000 conversations, that saves roughly $2100. The calculation does not account for output tokens or the cache. Cached input costs $0.075 per million, which is 20 times cheaper than regular input. Competitors claim different figures. Anthropic cited an 85% reduction in token use for its Tool search tool.

When it doesn't pay off?

There is no separate fee for the feature; tokens are billed at standard rates. But each server-side search means another model pass over the full context. That is almost like paying for the prompt a second time, minus the cached portion. Google itself gives a counterexample: 10 tools and a 30 000-token context. With deferred loading, the total comes to 92 830 tokens instead of 68 000, which is 36.5% more. Google advises keeping standard loading for libraries of up to about 20 stable tools.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Hands-On AI Engineering · Hamidun Academy

Build & Ship Production AI Agents Without Boilerplate Code

Start your free 72-hour trial at Hamidun Academy (academy.hamidun.com) — interactive lessons on Claude Code & Cursor, live Tech Radar, and production agent templates.

Start 72h Free Trial →Vibe-Coding Intensive
What do you think?
Loading comments…