Nous Research ships Tool Search for Hermes Agent, lifting accuracy 49–74% on Opus 4

Nous Research has released an update to Hermes Agent that adds Tool Search, a feature aimed at a long-standing problem for AI agents working with MCP (Model Context Protocol). With the new approach, the agent picks only the relevant tools instead of loading the full description of every available tool into context.

Context overload in MCP

When an AI agent talks to a toolset through MCP, the traditional approach requires loading the descriptions of all available tools into the model's context from the very start. With thirty, fifty or a hundred tools, each carrying detailed parameter descriptions and usage examples, this quickly becomes a problem: the context bloats, tokens run out sooner than they should, and the model itself can get lost in a sea of information.

Nous Research spotted this bottleneck and applied a classic information-retrieval technique: BM25 ranking. It is a full-text search algorithm that scores how relevant a document (here, a tool description) is to the user's query. Instead of loading the whole catalog, the system now searches for the best-matching tools and passes their schemas into context progressively.

How Tool Search works in practice

The mechanics are simple but effective. When the agent initializes, the system indexes the metadata of all available tools: their names, descriptions and purpose. When the user gives a command, the agent does not look at the full list. It first searches for the top-K tools relevant to the request:

• Step one: run a BM25 search over tool names and descriptions
• Step two: load brief schemas for the top results into the model's context
• Step three: if needed, expand the full parameters only for the selected tool
• Result: the context stays manageable and the agent chooses more accurately

This design solves several problems at once: the context shrinks, the agent runs faster, and, most importantly, tool-selection accuracy improves because the model is not distracted by irrelevant options.

Numbers from Anthropic

Nous Research tested Tool Search on standard Anthropic Evals benchmarks and got results worth noting. On Claude Opus 4, accuracy gains ranged from 49% to 74%, depending on the task type and the toolset. This is not statistical noise. It is a meaningful improvement, suggesting that narrowing the context to relevant tools really does help the model focus on the right choice.

Interestingly, the effect is most visible on large toolsets: the bigger the selection, the larger the gain from the smart filter.

What this means for AI agents

Tool Search is a small but telling example of how good context management can matter more than raw model power. Rather than simply enlarging the context window, it is sometimes enough to be smarter about what goes into that context. This applies not only to MCP but also to RAG systems, API integrations and automation systems where an agent must choose among many possible actions.

As AI systems grow more complex, such optimizations will become increasingly critical. The future belongs not to the biggest models, but to the ones that are smartest about what to look at.