EvoBelief teaches LLM agents to test hypotheses about the world: 83.3% vs 0% for baselines

An agent that knows how to act but misunderstands how the world works will still make mistakes. A new paper on arXiv proposes EvoBelief, a system in which a frozen LLM builds and revises a model of the environment in the form of falsifiable hypotheses. On ALFWorld without actor-side transition memory, it raises success rate over a comparable Frozen World Model by 23.9% with Claude Haiku 4.5 and by 31.3% with GPT-5.4. In Reactor Lab from DiscoveryWorld, the method completes 83.3% of Normal-level tasks and 58.3% of Challenge-level tasks, while each of the baselines tested scored 0%.

Most self-evolving agents improve their procedures: they accumulate experience, reflect, and build skill libraries. The EvoBelief authors choose a different axis of self-improvement: beliefs about environment dynamics. If an agent believes a door opens with a key, and the environment's rule has been changed, no skills will help. Worse, a faulty belief that was useful in one task leaks into others and corrupts them.

How the mechanism works

Knowledge about the environment is stored as a set of hypotheses with a limited scope of applicability. A hypothesis is born in two cases: when the outcome of an action turns out to be unexpected, or when the agent runs into an unknown that its decision depends on.

From there, a verification discipline applies. A rule does not enter predictions and plans until it is confirmed independently, either by later observations or by controlled experiments. The confirmation must not come from the same episode that produced the hypothesis. If a counterexample appears, the rule is narrowed or retired. This is how the agent responds to a change in the environment instead of clinging to an outdated picture.

The language model's weights are not touched. Only the external rule base changes, and every entry keeps a link to its evidence and to the decisions it changed. For engineers, this means you can open the base and see why the agent acted one way and not another.

Results on four benchmarks

The testing covered four sets of tasks: static dynamics, hidden drift, and scientific discovery. The paper reports the following results:

• ALFWorld without actor-side transition memory: +23.9% success rate over the Frozen World Model with Claude Haiku 4.5.
• ALFWorld, same setup with GPT-5.4: +31.3%.
• MiniGrid: the agent revises outdated rules and recovers after unannounced changes in dynamics.
• DiscoveryWorld, Reactor Lab, Normal level: 83.3% of instances completed.
• DiscoveryWorld, Reactor Lab, Challenge level: 58.3%.
• All evaluated baselines in Reactor Lab: 0% at both levels.

A note of caution: the percentages 83.3 and 58.3 look like 10 out of 12 and 7 out of 12, so the sample per level is probably small. The gap from zero is impressive, but statistical robustness should be checked on larger sets. The ALFWorld comparison is also tied to a specific configuration without actor-side transition memory, and the gain may differ in other setups.

Reactor Lab matters separately: the agent does not just memorize, it discovers and confirms quantitative laws. For scientific-discovery tasks, this is closer to real work than navigating rooms.

What to take into practice

If you are building agents for long-lived environments (internal tools, APIs that change without warning, simulations, lab automation), the idea applies even without this specific implementation. Store assumptions about the environment separately from skills and prompts. Give each rule a scope of applicability and a link to its evidence. Do not let a rule into planning until it has been independently verified, and narrow or delete it when counterexamples appear.

The second practical benefit is updating knowledge without fine-tuning. The model provider can change while the accumulated rule base stays, and it can be audited. The limitation is also obvious: controlled experiments require an environment that allows actions to be tried safely. In production, where a mistake costs money, you will need sandboxes or limits on which hypotheses may be actively tested.