Knowledge Conflict in RAG: Why You Need Probability of Context Correctness, Not a Rigid Algorithm

Knowledge

Conflict in RAG: Why You Need Probability of Context Correctness, Not a Rigid Algorithm

In May 2024, Google's search engine seriously advised users to add glue to pizza sauce and eat one rock a day — the model dutifully repeated a forum joke and a satirical article it had found online, even though its own knowledge should have been enough to spot the gag. That incident became a vivid illustration of a problem now dissected by researchers at the Posts and Telecommunications Institute of Technology in Hanoi: when a retrieval-augmented (RAG) model has two possible sources of truth — the retrieved text and its own memory — and they contradict each other, it's unclear which one to trust.

Why rigid rules don't work

Every existing approach resolves this conflict the same way: the direction of trust is chosen in advance, at the design stage. Some methods always defer to the retrieved text; others amplify its influence in proportion to how strongly it "pulls" the model's answer toward itself; still others rely solely on the model's memory. The authors proved mathematically that any such fixed rule behaves like a straight line on a graph plotting answer quality against the proportion of "correct" contexts — while the best possible outcome is described by a convex curve that lies above that line everywhere except at a single point.

In other words, any rigid rule is guaranteed to lose almost everywhere, and this isn't a matter of tuning parameters — it's a mathematical inevitability.

The researchers took a closer look at AdaCAD, a popular method that measures how much the model's answers diverge "with text" versus "without text" (using Jensen-Shannon divergence) and amplifies the context's influence proportionally to that divergence. The problem is that this metric is symmetric: it reacts the same way whether "the text is right and the model is wrong" or the reverse. Testing showed the divergence is, on average, the same regardless of whether the model is correct. As a result, precisely where the text is wrong, AdaCAD pushes the model hardest toward the incorrect answer — in 70% of cases, compared to 20% for a plain method with no special tricks at all.

How the solution works

To fairly test all these ideas, the authors built the GRACE benchmark based on questions targeting the Wikidata knowledge base. The trick is that for the same question, the context is deliberately corrupted or fixed in four different ways: left intact, stripped of the needed fact, swapped for an incorrect fact, or given both a correct and an incorrect version at once. This makes it possible to precisely measure what the model knows, what the text says, and what's actually true — independently of one another.

The solution consists of three steps:

• Knowledge notes. Before reading the context, the model formulates what it knows on the topic several times — once deterministically and five times with added randomness — as separate statements. Whatever the model repeats identically every time, it most likely genuinely knows; whatever changes from run to run, it's more likely guessing.
• Confidence calibration. A small trained model (a two-layer transformer) scores each such statement and assigns it a numerical confidence level, tuned so that the stated confidence tracks the real probability of being correct as closely as possible. This is enforced through a specialized reward function that penalizes false confidence more harshly than it rewards correct confidence.
• Final decision. A dedicated "judge" module (gradient boosting) compares the candidate answers derived from the text and from the model's memory, applies a predefined probability of "how often the context is actually right in this scenario," and either picks the better answer or honestly declines to answer if confidence is too low.

Results and what it means in practice

On the GRACE benchmark, the system beat the best of thirteen alternative methods tested, at every ratio of correct to incorrect contexts — gains reached 26 points and never dipped into a meaningful loss. The final average score was 45.3 for the Qwen3.5-35B-A3B model and 35.0 for Llama-3.1-8B, compared with 29.4 and 27.5 for the best competing methods. On the independent, pre-prepared ClashEval benchmark (with no tuning to fit it), the system outperformed the CK-PLUG and TCR methods by 12 and 26 points, respectively.

One important caveat: the advantage disappears in cases where the model simply doesn't know the answer itself — which makes sense, since the whole approach is built on comparing the model's knowledge against the text, not on magically guessing the truth. For engineers building RAG systems, the practical takeaway is simple: don't look for a single universally correct trust coefficient for retrieval. Instead, explicitly estimate the probability that context is correct for your specific use case, and build that estimate into the system as a tunable parameter — rather than hard-coding a rigid rule.