CausalAlign cures video models' "forgetfulness": geometry errors drop 20%

Video generators have learned to produce convincing clips, but they still suffer from an annoying ailment: the camera "drifts." After a few seconds, the trajectory strays from what was intended, scene geometry warps, and anything built on top of such video starts making mistakes. A team from PKU VaLuE Lab (Kai Zhu, Yiwu Zhong, and co-authors) proposes a method called CausalAlign. It cuts Wan2.2's geometric consistency errors by more than 20% and boosts the success rate of robotic action models by 10 points on RoboCasa365.

Where the model loses track of the camera

The core idea of the work is to diagnose first, then treat. The authors investigated how a diffusion video model represents, uses, and retains information about camera motion (ego-motion). They used two tools: linear probes, which show whether motion parameters can be extracted from a layer's activations, and causal interventions, which test whether that knowledge actually affects the output.

Both metrics behave the same way: they rise and then fall, both across network layers and across denoising steps. The model relies on ego-motion at intermediate stages but fails to retain it by the final output. The authors named this effect geometric state forgetting. In their view, this explains long-horizon drift: the network "understands" how the camera moved, but doesn't carry that understanding through to the final frames.

Delta tokens instead of the full scene

The cure is built on representation alignment. The researchers take a 3D foundation model that has strong knowledge of geometry and construct a space of delta tokens from its features. Each such token encodes the change between adjacent frames, rather than the frame itself. This matters because static backgrounds make up most of a video, and aligning features wholesale would drown the motion signal in redundant scene content. Deltas force the supervision to focus specifically on the dynamics.

The alignment is applied not across the whole network, but specifically at the points where ego-motion starts to fade. This way the method reinforces the weak link rather than rewriting the model entirely.

The results

Wan2.2 served as the base model, fine-tuned on the DL3DV and RealEstate10K datasets. Key results:

• Camera motion accuracy, geometric consistency, and temporal smoothness all improved noticeably;
• Geometric consistency errors dropped by more than 20% relative to the base model;
• Visual quality stayed at the same level, meaning the gains weren't bought at the cost of blur or artifacts;
• World action models (WAMs) using the fine-tuned video generator showed a 10-point gain in success on mobile manipulation tasks in the RoboCasa365 benchmark.

Research stack: Wan2.2 as the base video generator, a 3D foundation model as the source of "geometric ground truth," DL3DV and RealEstate10K as training data, and RoboCasa365 as the test of embodied tasks.

Why this matters beyond video

Interest in video models as "world models" for robots is growing: a robot predicts the future as video and then chooses an action based on it. If the predicted camera drifts, the action gets computed from a distorted future, and the error compounds systematically. This work demonstrates the link directly: improve geometry in the generator, and the agent's task success rate rises.

The practical takeaway for developers is this. Before scaling up data and parameters, it's worth probing your own model to find the layer and step where the needed signal disappears. Delta-alignment with an external 3D expert is cheap compared to retraining from scratch, and it's applicable to other video models, not just Wan2.2. The code and project page are open, so the method can be reproduced on your own data. The limitations are also clear: the results were obtained on specific datasets and a single benchmark, and transfer to longer clips and arbitrary scenes still needs to be verified.