Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseAPI / Product · ActionableGoogle AI Blog→ original

Gemini API Can Now Load Tool Schemas on Demand Instead of in Every Prompt

When an agent has hundreds of tools, the parameter schema for each one has to go out with every request. Token counts and latency climb, and the model gets worse at picking the right tool. Google has added a Tool Retrieval mode to the Gemini Interactions API: the model sees only a function's name and description, and loads the full schema when it needs it. For large tool libraries, this is a way to cut input tokens without a separate charge, but you should verify the savings on your own traffic.

AI-processed from Google AI Blog; edited by Hamidun News
Gemini API Can Now Load Tool Schemas on Demand Instead of in Every Prompt
Source: Google AI Blog. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceGoogle AI Blog ↗
Editor-in-Chief
Key metric #1Enable it with tool_search in the request and defer_loading: true on the tools
Key metric #2Works with Gemini 3.6 and newer, including all Gemini 4 models
No separate chargeyou pay for tokens plus a small fixed block

An agent handles the first five tools without trouble. At fifty, problems start: every function has a JSON schema of parameters, and all of it goes to the model with every request. The input-token bill grows, responses slow down, and the model more often picks the wrong function because there are too many candidates. Google has tried to fix this and launched Tool Retrieval in preview in the Gemini Interactions API.

How it works

The idea is similar to a table of contents in a book. The model always sees the name and a short description of each tool, so it knows what is available. The heavy part, the parameter schema, stays out of the context until it is needed.

To enable the mode, add the tool_search tool to the request and set defer_loading: true on the functions you want to hold back. Everything then happens inside a single API call. Say a user writes: "Order A-1042 arrived damaged, refund $40." The model requests the issue_refund tool by name, the API inserts its full schema, and the model builds a function call with the order number and the amount. All three steps are visible in interaction.steps: the search, the search result and the function_call itself. That helps with debugging: you can see right away which tools the model considered and whether the names and descriptions let it down.

From there it works like ordinary function calling. Your code executes the refund and sends a function_result back.

You do not need to build or host an index, because the search runs on the Gemini side. You can leave frequently used tools loaded up front and defer only the "long tail."

Cache, MCP and your own registry

There are three practical details. The first concerns the prefix cache. Deferred schemas do not sit at the start of the prompt, and the ones the model selects are added later in the conversation. As a result, the cached beginning does not change from turn to turn, and the cache benefit is not lost.

The second detail is about MCP. If you set defer_loading: true on a tool of type mcp_server, all functions of that remote server are loaded on demand. You can connect a large MCP server without listing its tools by hand.

The third detail is for those who already have their own tool registry, a vector index or a per-user permissions filter. For them there is a client mode: tool_search is declared with "execution": "client", and your application runs the search and returns the matching schemas in a function_result. The cost of this approach is one extra network round trip per search. In return, tools do not have to be declared in advance. When a user connects a new integration on the fly, its functions can be shown to the model in the middle of a conversation.

Pricing, limitations and what to do

There is no separate charge for Tool Retrieval. Input and output tokens are billed at the standard rate of the chosen model. But there is no free lunch here, and Google says so directly. The tool_search tool itself adds a small fixed block of input tokens. Each loaded schema is billed as input tokens for the rest of the conversation, just like a tool declared up front. Savings appear when the schemas the model never loads outnumber the ones it does. According to Google, that is the usual situation for large libraries, but the announcement gives no concrete savings figures. The company advises measuring the effect on your own traffic.

There are a few more limitations. The feature is still in preview, so the interface may change. It works only in the Interactions API and only with Gemini 3.6 and newer models, including all Gemini 4 models. If you have just a handful of tools, the benefit will be negligible, and the extra search step may add latency. Also, the model picks a tool by name and description, so weak or similar descriptions will pull in the wrong functions.

What should you do in practice? If you are building an agent with dozens of functions or connecting large MCP servers, a small experiment is worth running. Leave your hot tools loaded up front, defer the rest, and compare tokens, latency and the share of correctly chosen functions on real conversations. While you are at it, revisit your descriptions: they now decide whether the model finds the right tool.

Story

Key Questions Answered

How it works?

The idea is similar to a table of contents in a book. The model always sees the name and a short description of each tool, so it knows what is available. The heavy part, the parameter schema, stays out of the context until it is needed. To enable the mode, add the tool_search tool to the request and set defer_loading: true on the functions you want to hold back. Everything then happens inside a single API call.

What are the key details on Cache, MCP and your own registry?

There are three practical details. The first concerns the prefix cache. Deferred schemas do not sit at the start of the prompt, and the ones the model selects are added later in the conversation. As a result, the cached beginning does not change from turn to turn, and the cache benefit is not lost. The second detail is about MCP. If you set defer_loading: true on a tool of type mcp_server, all functions of that remote server are loaded on demand.

What are the key details on Pricing, limitations and what to do?

There is no separate charge for Tool Retrieval. Input and output tokens are billed at the standard rate of the chosen model. But there is no free lunch here, and Google says so directly. The tool_search tool itself adds a small fixed block of input tokens. Each loaded schema is billed as input tokens for the rest of the conversation, just like a tool declared up front. Savings appear when the schemas the model never loads outnumber the ones it does.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…