WorldBridge Model Lifts Success Rate on Unseen Tasks from 25.07% to 70.70% on RoboTwin 2.0

Researchers have introduced WorldBridge, a World Action Model (WAM) that predicts how a scene will evolve in video and translates that prediction into commands for a robot. The central question of the work is what helps a robot more with tasks it never saw in training: more video or more labeled actions. On RoboTwin 2.0, a simulator of tasks for dual-arm robots, the success rate on such new tasks rose from 25.07% when training only on familiar data to 55.90% after adding video of those tasks, and to 70.70% after adding just one demonstration with actions per task. The model itself also posted the best results among the compared methods on several manipulation benchmarks.

What video contributes and what actions contribute

The experiment used three training regimes. In the first, the model saw only "video + actions" pairs for tasks from the original training set. On new tasks (out-of-distribution, meaning outside the training distribution), it succeeded 25.07% of the time — a nonzero but weak result.

In the second regime, the researchers added video of the new tasks to training, but without action labels; action labels remained only for the original tasks. Success rose to 55.90%. The authors suggest the model draws on an inverse dynamics ability it has already learned: it recovers the action that led to a change in the scene, and applies that skill to the new video.

In the third regime, the same videos were supplemented with one demonstration with actions for each new task. Success climbed to 70.70%. The authors describe a division of roles: video widens the range of tasks the model can "imagine," while even a small number of labeled actions strengthens the bridge from image to control. The finding is consistent with DreamZero and Zero-WAM, which already used video priors to perform unfamiliar tasks and to transfer skills from video without actions.

How WorldBridge works

The architecture is a Mixture-of-Transformers with two separate experts: one for video and one for actions. Each has its own parameters, feed-forward networks and decoder, and they exchange information through shared self-attention. The video expert additionally receives a text instruction via cross-attention. A video VAE and a T5 text encoder encode the context, and training follows a shared flow matching objective with independent noise levels for video and actions.

The model's features include:

• Structured attention mask. Information flows from the observed context to future video representations and then to actions, so the predicted scene dynamics explicitly condition control.
• Visual memory at two scales. Short-term memory stores local motion; long-term memory tracks task progress.
• Action forcing. Within a single action block, different elements are assigned independent noise levels during training, and the action expert learns to rely on temporal context corrupted to varying degrees.

The researchers also describe a unified data pipeline. Robot trajectories and egocentric demonstrations of human hands are converted to a shared 46-dimensional action representation. Per-dimension validity masks preserve partial labels for platforms with different control interfaces. Robot data goes through kinematics and temporal-consistency checks, while hand trajectories are standardized in a common coordinate system and filtered by geometric and motion constraints.

Results and limitations

Among the compared methods, WorldBridge shows the best results: an average success rate of 96.23% on RoboTwin 2.0 in the full setting, 80.9% in the clean-to-random setting, 82.8% on RoboCasa GR1 TableTop and 57.6% on DOMINO Level-1. On RMBench, the model matched the best result in the authors' comparison, 83.0%. It was also evaluated on SimplerEnv-WidowX and RoboDojo. On the real robots AgiBot G2 and ARX AC-1, the same jointly pretrained checkpoint was fine-tuned for each platform; the tests covered long action chains, tasks requiring memory, spatial positioning and bimanual coordination.

Some caveats apply. The key transfer figures come from a simulation benchmark, and the leadership claim is made "among the compared methods," meaning within a set chosen by the authors themselves. For practitioners, the main takeaway is this: cheap video without labels can deliver most of the gain when transferring to new tasks, while a few labeled demonstrations are worth adding selectively to lock in the result.