Nous Research released Tool Search for Hermes Agent: accuracy improved by 49–74% with Opus 4

Nous Research introduced an update to Hermes Agent — adding a Tool Search feature that solves a longstanding problem for AI agents working with MCP (Model Context Protocol). The new approach allows the agent to select only relevant tools instead of loading the full description of all available tools into context.

Context Overload in MCP

When an AI agent interacts with a tool system through MCP, the traditional approach requires loading descriptions of all available tools into the model's context right from the start. If there are thirty, fifty, or a hundred tools, and each has detailed descriptions of parameters and usage examples, this quickly becomes a problem: the context balloons, tokens run out sooner than needed, and the model itself can get lost in seas of information. Nous Research noticed this bottleneck and decided to apply a classical approach from information retrieval — BM25 ranking.

This is a full-text search algorithm that accounts for the relevance of a document (in this case, a tool description) to a user's query. Instead of loading the entire catalog, the system now searches for the most suitable tools and passes their schemas into the context progressively.

How Tool Search Works in Practice

The mechanics are simple but effective. When initializing the agent, the system indexes the metadata of all available tools — their names, descriptions, and purposes. When a user gives a command, the agent doesn't look at the full list, but first searches for the top-K tools relevant to the query:

• First step: perform BM25 search on tool names and descriptions
• Second step: load brief schemas for top results into the model context
• Third step: if needed, reveal full parameters only for the selected tool
• Result: the context remains manageable, the agent chooses more accurately

This arrangement solves several problems at once: the context size is reduced, work speed is improved, and most importantly — tool selection accuracy is increased, because the model isn't distracted by irrelevant options.

Numbers from Anthropic

Nous Research tested Tool Search on standard Anthropic Evals benchmarks and got results worth noting. On the Claude Opus 4 model, accuracy improvement ranged from 49% to 74% depending on the task type and tool set. This is not merely statistical noise — it's a meaningful improvement that shows that narrowing the context to relevant tools truly helps the model focus better on the right choice. Interestingly, the effect is most noticeable precisely with large tool sets — the more options to choose from, the greater the gain from the smart filter.

What This Means for AI Agents

Tool Search is a small but telling example of how proper context management can be more important than raw model power. Instead of simply increasing the context window, sometimes it's enough to approach more intelligently what information to load into that context. This applies not only to MCP, but also to RAG systems, API integrations, automation systems where the agent must choose from many action options. As the complexity of AI systems grows, such optimizations will become increasingly critical. This means the future belongs not to the largest models, but to the smartest ones at choosing what to look at.