HeirCommit Filters Out Up to 83% of Harmful Updates in Self-Evolving LLMs

Researchers have identified a weak spot in self-evolving language models: the choice of which model version to pass into the next round. On verified tasks, most updates do not raise the probability of correct answers, yet the R-Zero, Agent0 and Skill Self-Play loops inherit all of them. The HeirCommit framework cuts the share of harmful accepted updates by 68–83% relative to random inheritance at the same acceptance rate. The EvoHeir add-on delivers the best average results in math and general reasoning across four base models. Its average math score is 5.3% higher than that of Agent0, the strongest baseline.

Where self-evolution breaks down

The self-evolution scheme works like this: the model generates tasks, learns from them, and enters the next round as an updated version. Each round requires a decision about which model version becomes the heir. In existing loops, that decision is made through the task. If a task looks learnable, it is admitted to training, and the resulting update is accepted wholesale.

The authors call the problem the selection-to-update gap. A task that looks learnable does not necessarily produce a useful shift in the weights. By their estimate, most updates on verified tasks do not make correct answers more likely. The loop inherits them anyway, and errors accumulate from round to round.

How HeirCommit and EvoHeir work

HeirCommit moves the decision from the task to the model itself. The updated version is frozen and tested on fresh evaluation questions. The result determines whether to inherit the new model or keep the current one. Fresh questions matter because scores on the questions used for selection come out inflated. The authors call this effect selection-induced optimism and describe it analytically.

EvoHeir goes further and splits inheritance into two channels: parameters and experience. A model may fail the check, but the tasks and trajectories it learned from remain useful. Current and archived experience is used to fix errors and retain successful solutions. Final confirmation comes only after this step. In the authors' wording, useful experience is preserved even when the model that produced it is not inherited.

The theoretical part decomposes regret into three components:

• selection error, when a bad update is accepted or a good one is rejected;
• absorption error, when the model poorly absorbs what it has been shown;
• confirmation error, when the final check introduces its own noise.

The numbers

• The share of harmful updates among those accepted is 68–83% lower with HeirCommit than with random inheritance at the same acceptance rate.
• EvoHeir was tested on four base models.
• It posts the best average results in math and general reasoning among the compared methods.
• Gain in average math score: 5.3% relative to Agent0, the strongest baseline.

Comparing against random inheritance at a fixed acceptance rate makes the result more credible. The gain cannot be explained by the method simply becoming stricter. The only question is which updates get through. The 5.3% gain is stated as relative, and the available description does not make clear which benchmarks and models went into the average.

What to take away

If you are building a self-training loop for your own model or agent, the work suggests three practical conclusions. First, do not treat a task's admission as a guarantee of a useful update: check the resulting model, not just the material it trained on. Second, keep a separate set of fresh questions for the inheritance decision. If you reuse the questions that drove selection, the score will be inflated. Third, do not throw away the experience of rejected versions. The trajectory archive can be used to fix errors and lock in successes, and accepting the candidate weights is not required for that.

The limitations are also clear. Fresh evaluation questions cost compute and data, and implementation details and the full list of benchmarks are best checked in the paper itself. But the core idea of separating the choice of the future model from the choice of the training task applies to any self-evolving loop.