NVIDIA unveils NeMo Retriever agentic search for complex enterprise data

NVIDIA has introduced a new agentic pipeline for NeMo Retriever, a search system that goes beyond comparing semantic similarity. Instead of a single query, it runs a loop of searching, evaluating intermediate results and refining its strategy, which helped the solution take first place on the ViDoRe v3 leaderboard and second on BRIGHT.

Why a single search is not enough

Classic dense retrieval works well when it is enough to find documents that resemble the query in meaning. In enterprise scenarios that is often not enough: documents can be visually complex, queries compound, and answers scattered across several sources. These tasks need more than an embedding match. They require the ability to break a question into parts, test hypotheses and change the direction of the search several times.

NVIDIA describes this as a gap between two types of systems. Retrievers can quickly scan huge volumes of data, but they barely reason. Large language models can plan and draw logical conclusions, but cannot process millions of documents at once. Agentic retrieval is meant to close this gap by combining both approaches in a single loop.

How the loop works

The pipeline is built on the ReACT architecture. The agent does not receive a task in a "one query, one result" format. It acts step by step: it thinks, calls the retrieve(query, top_k) tool, analyzes what it found and decides what to do next. The final answer is assembled through a separate tool, final_results, which returns a list of the most relevant documents.

According to the team, several useful patterns emerged naturally in the process:

• generating more precise queries as new facts appear;
• constant rephrasing until the system finds a useful signal;
• splitting a complex question into several simple subtasks;
• re-ranking the retrieved documents before the final selection.

If the agent hits the step limit or the context length limit, the pipeline does not simply cut off blindly. Reciprocal Rank Fusion serves as a safety net: documents receive a final score based on their positions across different search attempts, and the system still returns a meaningful set of results.

Where the pipeline came out ahead

The main result is not only the leaderboard position but also versatility. The same pipeline, without changing the base architecture, took first place on ViDoRe v3 with an NDCG@10 of 69.22 and second place on BRIGHT with an NDCG@10 of 50.90. The first benchmark matters for visually rich and varied enterprise documents; the second for tasks that require more multi-step reasoning.

The authors separately compared their approach with more specialized solutions. For example, INF-X-Retriever leads on BRIGHT with a score of 63.40, but on ViDoRe v3, in the same configuration with nemotron-colembed-vl-8b-v2, it scored 62.31, even lower than plain dense retrieval on the same embedding model, which scored 64.36. NVIDIA uses this comparison as an argument for a generalizable approach: an agentic loop transfers better between different types of tasks than pipelines tuned for a single benchmark.

Also notable is that the team reworked the infrastructure for speed. The retriever was first moved into an MCP server, which makes sense for giving an LLM access to external tools. In practice, though, it added extra network calls, a separate process, the risk of silent configuration errors and crashes under load. In the end, the MCP setup was replaced with a thread-safe in-process singleton retriever: the model and embeddings are loaded once, access is synchronized through a lock, and the retrieve() interface stays the same. This removed a whole class of operational problems and sped up experiments.

The price of autonomous search

NVIDIA states plainly that this quality comes at a cost. Agentic retrieval is noticeably slower and more expensive than ordinary dense retrieval. On ViDoRe v3, a single query took 136.3 seconds on average, required about 760 thousand input tokens and 6.3 thousand output tokens, and the agent made 9.2 search calls on average. For real-time tasks this is a heavy profile, especially under mass load.

The team also compared closed and open models. On ViDoRe v3, the setup with Opus 4.5 performed best, but switching to the open-weight gpt-oss-120b caused only a moderate drop in quality, from 69.22 to 66.38. On BRIGHT the gap was larger, which indicates that complex reasoning tasks depend on more powerful frontier models.

NVIDIA's next step is to try to transfer these agentic patterns to more compact, specialized open models, in order to cut cost and latency without a large loss of quality.

What this means

Search over enterprise data is quickly moving away from the "enter a query, get similar documents" scheme. NVIDIA shows that the next level is an agent that can search iteratively, change tactics and combine reasoning with retrieval. For now this approach is expensive and slow, but for complex high-stakes scenarios it already looks like a working architecture rather than a lab experiment.