DeCert Teaches AI Judges to Explain Their Verdicts: An 8B Model Scores 76.2 on RM-Bench

A generative reward model is an LLM that compares two answers, explains in words which one is better, and delivers a verdict. Such judge models underpin RLHF pipelines and automated evaluation. They have a weak spot: a correct verdict does not mean the reasoning behind it was correct. The model may have guessed, latched onto an irrelevant detail, or written a rationale that does not lead to its conclusion at all. A team of Guiquan Sun, Bing Bai, Martin Renqiang Min and Dongjin Song proposes DeCert, a framework that trains judges to write convincing rationales, and does so without reference explanations. The 8B model DeCert-Qwen3-8B scores 76.2 on RM-Bench and 52.0 on JudgeBench.

Why a Correct Verdict Is a Weak Training Label

The standard recipe for training a judge is simple: sample several judgments and reward those whose verdict matches the label. But two judgments with the same correct verdict can differ greatly in quality. One analyzes the facts and logic of the answer; the other offers generic phrases. The classic scheme treats them as equal, so the model gets no signal about which one is better.

Some existing approaches address this with reference explanations: the judge's rationale is compared against a gold standard. The authors point to two problems. Such references are expensive to obtain. And a match with the reference does not prove that the generated rationale actually supports the verdict it reaches.

How DeCert Scores the "Decision Certificate"

DeCert checks the rationale itself: how valid it is and how well it agrees with the verdict. These scores become a preference-based training signal that accounts for both verdict correctness and the quality of the argumentation. A rationale is scored along three axes:

• soundness: whether the arguments it presents are correct and well-founded;
• coverage: completeness, i.e. whether the essential aspects of the compared answers are addressed;
• decision support: whether the rationale actually leads to the verdict that was issued.

The key move concerns the case where several samples produce the same correct verdict. Previously they had to be treated as equal. Now the difference in the quality of the "certificate" makes it possible to build preference pairs at the rationale level: strong against weak. The model learns to tell a solid explanation from one that is formally correct but empty.

The Numbers and Robustness to Answer Order

The method was tested on two families of 8B models and several benchmarks, both in-distribution and out-of-distribution. According to the authors, DeCert consistently outperforms strong generative judge baselines while also improving rationale quality. The main results:

• DeCert-Qwen3-8B: 76.2 on RM-Bench;
• DeCert-Qwen3-8B: 52.0 on JudgeBench;
• gain in position-consistent accuracy of up to +14.8 points;
• improvement on three measures: judgment accuracy, rationale quality and position robustness.

The last metric deserves separate attention. Position bias is a well-known problem of LLM judges: the verdict depends on which slot an answer occupies, not on its quality. Position-consistent accuracy counts a success only if the model picks the correct answer under both presentation orders. A gain of up to 14.8 points shows that requiring quality rationales also reduces such artifacts. A judge that is forced to explain why one answer is better is less likely to lean on a random signal such as position.

What You Can Take Into Practice

The code is available on GitHub in the UConn-DSIS/DeCert repository, and the paper is on arXiv under number 2610.27528. The idea applies directly to teams building LLM-as-a-judge systems or training reward models for RLHF and RLAIF. If you need a judge to produce not just scores but readable explanations (for dataset audits, agent debugging, annotation), it makes sense to set a separate training objective for rationale quality rather than stopping at verdict agreement. When sampling several judgments per example, it is worth examining those with identical verdicts too: that is where the signal about reasoning quality hides. It is also worth measuring position-consistent accuracy separately, not just plain accuracy: it shows how far a judge can be trusted.

The limitations are visible too: the authors' models are small, 8B, and how the approach scales remains to be tested. But the ability to improve judges without expensive reference explanations makes the method interesting for anyone who wants reliable, explainable evaluations of LLM answers.