Stateful Harness Lifts World Model Accuracy on Hidden State from 32% to 92%

The authors of a paper in Hugging Face Daily Papers have proposed Stateful Harness, a framework for world models that separates world evolution from video rendering. On the new WorldState-Bench benchmark, which has 50 counterfactual pairs, the best of 11 video backends, prompted with rules only, produces the required endings for both cases in 32% of pairs. The full harness with the same frozen MiniMax-H3 reaches 92%. The video model itself was not fine-tuned.

Where world models break

Modern world models can generate video from a single image, taking the user's action into account. Many focus on camera movement: you can walk through the generated scene and look around. But navigation says nothing about whether an action obeys stable rules. When the outcome depends on a state that is not in the current frame, the model has to guess. For example, whether a door opens depends on whether it is locked, and the frame does not show that. The video model picks the most plausible outcome, not the one the world's rules require.

The authors call this the hidden-state problem. It becomes more noticeable the longer the session runs: each next turn builds on what happened before, and a purely video-generative model has no reliable place to store that.

How Stateful Harness works

The idea is to split the roles. The harness runs the world and its rules, while the video model only shows the result. A turn works like this:

• the harness stores an evolving world state according to the defined rules;
• it resolves the action the user chose and compiles the resulting event into a text instruction;
• a preview clip from an editable 3D scene provides a spatial and camera cue;
• the frozen video model renders the event;
• the generated clip is used to update the world state before the next turn.

The last step closes the loop: the next turn builds on what actually appeared on screen, not just on the internal record.

The benchmark and the numbers

To test rule adherence, the authors built WorldState-Bench. It consists of 50 counterfactual twin pairs. Each pair shares an initial frame and an action, but has a different hidden state and a different required outcome. A pair counts only if the model produced the correct endings in both cases. As a result, a model that always draws one typical outcome is guaranteed to fail half of every pair.

Key results:

• 11 video backends were tested with rules-only prompts;
• the best of them produces the required endings in both cases in 32% of pairs;
• MiniMax-H3 with the required endings stated explicitly rises from 32% to 64%;
• the full harness with the same frozen MiniMax-H3 reaches 92%.

The gain splits roughly in half. The first 32 points came from the simple fact that the rule was turned into a concrete outcome in the prompt. The other 28 points came from the rest of the harness: the 3D preview, state carried between turns, and feedback from the clip. The video model did not get smarter; it was simply no longer asked to do work beyond its abilities.

What to do with this in practice

For teams building interactive simulators, game prototypes, or training environments for agents, the takeaway is down-to-earth: keep the world's logic in code instead of asking the video model to "remember" it. Rules, inventory, locks, and counters are described explicitly, and the generator receives a ready-made instruction. This also simplifies debugging: you can trace an error in an outcome to a rule rather than to the weights.

Caution is also warranted. The benchmark is small, just 50 pairs, and the authors built it themselves. The 92% figure shows that the approach works on this set, but it does not guarantee the same result in open-ended scenes with hundreds of rules. Latency, per-turn cost, and the difficulty of writing rules for new worlds are not disclosed in the description provided, and for a product these are decisive parameters.

A useful step you can take now: build your own counterfactual pairs for your task and run your current video backend through them. If the result is close to 32%, the problem is most likely the architecture, not the choice of model.