Box integrates Nvidia Nemotron into agent search: 4x faster with 90% fewer tokens

If you have built agents on top of corporate files, you know the pain: an agent receives a question, searches the repository, and pulls half a document into its context just to find a single paragraph. It is slow, expensive, and prone to errors. Box has addressed the problem at the platform level by integrating Nvidia Nemotron embeddings into the product's core.

What was unveiled

At its developer conference, Box introduced Box Content Retrieval. Instead of giving agents entire documents, it returns the passages most relevant to a specific query or task. CEO Aaron Levie described it this way: an agent no longer needs to “read every document”; it receives the passages it needs.

The stated figures come from Nvidia. Kari Briski, vice president of generative AI, expects responses to be four times faster, with 20% fewer errors and 90% fewer tokens used. The feature will become available to all Enterprise Advance customers early next year.

How it works in practice

Imagine a lawyer asking an agent to find every limitation-of-liability clause across a hundred contracts. Conventional keyword search will miss clauses phrased differently or written in another language. Embeddings work differently: they convert text into a numerical representation of its meaning. This allows semantically similar clauses to be found even when they use different wording.

Nemotron Embed is a family of open-weight models that use an LLM to preprocess data. The agent then uses the output of that processing during search. According to the article, the July Nemotron release topped RTEB, a benchmark that measures retrieval quality. Major labs have often underestimated models of this kind, while Nvidia has made a significant bet on them.

The token savings make sense. If the context contains one relevant passage instead of a long file, the model reads less text. That improves speed and cost, while less noise in the context usually means fewer errors.

Limitations and pitfalls

The models alone are not enough. Although Nemotron is available for free, Box relies on its proprietary metadata and access-control system, making the solution difficult to recreate from scratch. Levie says the company has packaged roughly five different technologies into one simple API. For Box, that is a competitive moat; for teams building something similar in-house, it is a warning about the amount of work involved.

There is also a caveat to the figures. The promises of 4x performance, 20% fewer errors, and 90% fewer tokens come from Nvidia and the vendor itself, while the product has not yet been released. The measurement conditions are unknown, so teams will need to validate the claims on their own data. The feature is also available only on the highest-tier plan, meaning it remains a future option for other Box customers.

What to do in practice

First, if you are building agents with access to documents, treat retrieval as a separate layer rather than a minor feature attached to the model. In most cases, it determines both the token bill and the quality of the answers.

Second, you can already test the open Nemotron embedding models on your own corpus and compare them with your current system. Pay particular attention to multilingual data, where keywords often fall short.

Third, product teams should remember that access rights and metadata are part of search quality. An agent that finds the right information but fails to account for who is allowed to see it is useless to a business. Box considers this layer its advantage, and it should also be a key consideration when choosing between a ready-made platform and an in-house system.