Habr: 6 Reasons Your RAG Agent's Memory Fails, and How to Test Retrieval

When a RAG agent gives an off-target answer, the language model is the first suspect. In practice, the culprit is more often search: if the right fragment never reaches the context, even a strong LLM will either invent an answer or say that no data is available. The Habr article covers six causes of such failures and a method for diagnosing retrieval. The announcement of the post gives only a definition of the technology; the list of causes itself is hidden behind a cut. So below is a map of typical failures, which engineers usually check in this order.

RAG (Retrieval-Augmented Generation) connects a model to an external knowledge base. The agent does not rely only on what it memorized during training. It first looks up information in corporate documents, reference materials, or the internet, and writes its answer based on what it finds. The pipeline has at least five stages: document chunking, vectorization, indexing, search, and context assembly. An error at any of them looks the same: the agent "forgot."

Where search usually breaks

The first risk zone is chunking. Chunks that are too small lose context, while chunks that are too large blur the meaning of the vector. A table cut in half, or a definition separated from its term, will not be found by any query.

The second is the embedding model. If it handles Russian, narrow terminology, or code poorly, semantically similar texts end up far apart in the vector space.

The third is index freshness. A document was updated, but reindexing was never run, so the agent confidently quotes last year's version of a policy. For agent memory, this is the most common source of "outdated" answers.

The fourth is query wording. The user writes "how do I request time off?", while the document in the database is titled "Procedure for granting annual paid leave." Query rewriting and splitting a complex question into sub-queries help.

The fifth is ranking. Pure vector search is poor at finding part numbers, names, and abbreviations. A hybrid setup with BM25, plus a reranker over the top candidates, usually delivers a noticeable gain in accuracy.

The sixth is context assembly. Models use facts buried in the middle of a long prompt less effectively. Duplicates, extra fragments, and a poor ordering reduce quality even if retrieval worked correctly.

How to diagnose retrieval

The main principle is to test search separately from generation. To do this, build a reference set of questions, each linked in advance to its source fragment. A few dozen or a couple of hundred examples are enough, but they should reflect real user queries, not ones invented by developers.

Next, calculate the metrics:

• recall@k: the share of questions for which the right fragment landed in the top-k results. If this fails, there is no point looking further;
• precision@k: how much of what was retrieved is actually useful, that is, how much noise goes into the prompt;
• MRR: the average position of the first correct result;
• nDCG: the quality of result ordering when several fragments are relevant;
• search and reranking latency, because accuracy bought with seconds of waiting is often unacceptable for an agent.

If recall is high but the answers are bad, the problem lies further down the pipeline: in context assembly or in the model itself. If recall is low, start with chunks and embeddings.

Practical order of steps

First, build the reference set and record baseline numbers. Then change one parameter at a time: chunk size, embedding model, adding BM25, the reranker, the number of fragments passed on. Check every change against the metrics, not against the impression left by a few successful conversations.

Set up monitoring separately: log the query, the retrieved fragments, and their scores. When a user complains about a wrong answer, first check whether the right document was in the results. Set up regular reindexing and verify that updated documents actually made it into the index. And do not try to fix retrieval with the prompt: asking the model to "answer more accurately" will not bring back a fact that is not in the context.