WorldBridge Lifts Robot Task Success Rate from 25.07% to 70.70% on RoboTwin 2.0

The WorldBridge researchers worked out what helps a robot perform tasks that were not in its training, and built their own model around that finding. On the RoboTwin 2.0 simulation benchmark, the success rate on new out-of-distribution tasks (that is, tasks outside the training set) was 25.07% without additional data. After the researchers added video of those tasks without action labels, it rose to 55.90%. With one labeled demonstration per task, it reached 70.70%. The work concerns World Action Models (WAM): models that first predict how a scene will change in video, then turn that prediction into commands for a manipulator.

What the video contributed and what the actions contributed

The authors separated two skills. Video generalization is the ability to predict how an unfamiliar task should unfold. Action generalization is the ability to translate that prediction into executable movements. The translation is handled by a "bridge" from video to action, which works as an inverse dynamics model (IDM): given a change between frames, it reconstructs the movement that produced it.

When the model is trained only on "video plus action" pairs from the original set of tasks, it succeeds on new tasks 25.07% of the time. Adding video of the new tasks without action labels raises that to 55.90%. Adding just one demonstration with actions for each new task brings success to 70.70%.

The authors explain the result this way: a bridge trained on the original tasks already knows how to turn visual transitions into movements. Video expands the set of transitions the model can predict, while the few labeled examples make the bridge itself more accurate.

How the model is built

WorldBridge is built on a Mixture-of-Transformers architecture. It has two separate experts, one for video and one for actions, each with its own parameters. They exchange information through joint self-attention and are trained with a shared flow matching loss function. A T5 text encoder encodes the natural-language instruction, and a video VAE encodes the observations.

Key design decisions:

• Structured attention mask. Information flows from the observed context to the future video and only then to the actions, so the predicted scene dynamics explicitly drive the movement.
• Two-scale visual memory. Short-term memory stores data about local motion; long-term memory stores data about task progress.
• Action forcing. During training, actions within a block are assigned independent noise levels, and the model learns to predict them from context with varying degrees of corruption.

Data and results

For pretraining, the authors built a unified data pipeline. Trajectories from different robots and egocentric demonstrations with human hands are converted to a common 46-dimensional action representation. Per-dimension validity masks preserve partial labels when a source lacks some of the control signals. Robot trajectories go through checks for kinematic and temporal consistency, while hand trajectories are converted to a single coordinate system and filtered by geometric constraints and motion parameters.

The authors claim that WorldBridge achieves the best result among the compared methods on several benchmarks: 96.23% on RoboTwin 2.0 with the full dataset, 80.9% in the C2R (clean to random — from clean scenes to random ones) setting, 82.8% on RoboCasa GR1 TableTop, and 57.6% on DOMINO Level-1. On RMBench, the model matches the best result in the comparison at 83.0%. The authors also evaluated it on SimplerEnv-WidowX and RoboDojo.

The same pretrained checkpoint was fine-tuned for the real-world platforms AgiBot G2 and ARX AC-1. There, the researchers tested long action sequences, tasks requiring memory, spatial positioning, and two-handed manipulation.

The figures of 25.07%, 55.90%, and 70.70% come from simulation, and the claim of being "better than alternatives" applies only to the methods the authors chose for comparison. For teams collecting data for robots, the takeaway is practical: collect unlabeled video of new tasks first, and spend a limited action-labeling budget on one demonstration per task.