Runway Open-Sources Praxis-1: A Video Model That Controls a Robot and Solves 46.2% of Tasks in Nvidia's Benchmark
Runway has open-sourced the code and weights for Praxis-1, a model that calculates robot movements from camera frames and a command such as "put the banana on the plate." In Nvidia's RoboLab-120 simulation (120 tasks: sorting, grasping, stacking), it solved 46.2% of tasks against 42.9% for FLUX 3 Action. On a real Franka arm, it succeeded in 88 of 100 attempts versus 80 for the competitor. Runway ran these tests itself, so run the model on your own tasks before deploying it.
AI-processed from The Decoder; edited by Hamidun News
Robots are hard to teach for a simple reason: getting a manipulator to carefully pick up a banana or close a drawer requires hundreds of hours of expensive recordings in which a human manually controls the machine. Runway, known for video generators such as Gen-4.5, took a different approach and released Praxis-1 as open source.
It is a model that calculates robot movements from camera frames and a text command, drawing its knowledge of the physical world from training on video. In Nvidia's RoboLab-120 test (120 simulated tasks involving sorting, grasping, and stacking objects), it completed 46.2% of tasks, while the closest comparable model, FLUX 3 Action, completed 42.
9%. The weights and code are available under the Apache 2.0 license, which permits commercial use.
How a Video Model Became a Controller
Praxis-1 takes as input camera frames, a text instruction ("put the banana on the plate"), and data on the robot's own position. From these, the model predicts how the scene will change and computes the required movements based on that prediction. When new frames arrive, it adjusts its control on the fly.
The logic is easy to grasp through a human example. You have seen thousands of clips where objects fall, roll, and collide, so you roughly know what will happen if you nudge a mug. Praxis-1 was first trained on ordinary videos so it could learn how objects move and interact. Runway then fine-tuned it on videos of people performing actions and on robot data. The company calls this class of systems a World Action Model: first predict the scene, then act.
Runway hopes the approach will reduce reliance on specially collected robot datasets. Still, special data is required anyway: for the Franka arm, the team used about 277 hours of recordings from the public DROID dataset, and for a two-armed robot, between 1,400 and 2,600 hours. Whether the model needs less robot data than analogues without video pretraining is unknown, since there is no direct comparison.
What the Tests Showed
On the Nvidia RoboLab-120 leaderboard, Praxis-1 achieved a 46.2% success rate, meaning slightly less than half of the tasks were completed fully. FLUX 3 Action, which is also built as a World Action Model, scored 42.9%. That is the best result in its category, but in the overall ranking the model is only fifth: two AI agents and two robot models with a different architecture rank above it.
On a real Franka arm, after fine-tuning on DROID, the team compared the same two models on ten tasks, including "put the banana on the plate" and "close the drawer." With ten attempts per task, Praxis-1 succeeded 88 times out of 100, and FLUX 3 Action succeeded 80 times.
The team also separately tested transfer to an unfamiliar robot: a mobile platform with a manipulator, on which Google once developed RT-1. Praxis-1 had no recordings from that robot in its earlier training. After fine-tuning on roughly 95 hours of data, the model achieved an average simulated success rate of 80% across four task types, from grasping soda cans to placing an apple in a box. The original RT-1 scored 52.4% on the same test. An important caveat: Runway conducted all the tests itself, and there has been no independent verification.
Since late September, the model has been tested by Standard Bots, Noble Machines, and Ultra, on a humanoid, an industrial arm, and a two-armed warehouse machine.
Limitations and What Developers Should Do
The base model is not suitable for new robots out of the box: it must be fine-tuned. Adapted versions already exist for popular platforms. The model can be downloaded from Hugging Face after registration and acceptance of the access terms, and the code is on GitHub.
The requirements are substantial: the standard version needs a graphics card with 80 GB of memory. According to Runway's estimates, the compressed version will fit on a 32 GB RTX 5090, but the actual peak memory consumption has not yet been measured in the GitHub documentation.
The company does not hide the weak spots. The model sometimes confuses similar objects and handles vague commands poorly. Long tasks must be broken into steps manually, because the model cannot reliably plan complex action sequences on its own. Runway sees Praxis-1 primarily as a fast executor of movements that could later be connected to a general-purpose language model: the language model would break a task into steps, and Praxis-1 would carry them out.
The practical takeaway: if you are building a robotics stack or researching manipulators, you now have an open, commercially usable foundation that can be compared with other robot control models. But for now, all the figures are Runway's own claims, so run the model on your own tasks and hardware before relying on the stated percentages.
- Nov 21, 2026llama.cpp Hits Version 1.0.0: Faster on NVIDIA, Intel, and ARM, SSRF Holes Closed
- Nov 21, 2026AWS and NVIDIA show how to split a coding agent's requests between Claude Opus 5.5 and Nemotron
- Nov 21, 2026Former OpenAI Researcher Claims to Find Traces of Banned Blackwell Chips in Six Chinese AI Models' Weights
- Nov 21, 2026Former OpenAI Researcher Finds Traces of Banned Blackwell Chips in Six Chinese Models
Key Questions Answered
How a Video Model Became a Controller?
Praxis-1 takes as input camera frames, a text instruction ("put the banana on the plate"), and data on the robot's own position. From these, the model predicts how the scene will change and computes the required movements based on that prediction. When new frames arrive, it adjusts its control on the fly. The logic is easy to grasp through a human example. You have seen thousands of clips where objects fall, roll, and collide, so you roughly know what will happen if you nudge a mug.
What the Tests Showed?
On the Nvidia RoboLab-120 leaderboard, Praxis-1 achieved a 46.2% success rate, meaning slightly less than half of the tasks were completed fully. FLUX 3 Action, which is also built as a World Action Model, scored 42.9%. That is the best result in its category, but in the overall ranking the model is only fifth: two AI agents and two robot models with a different architecture rank above it.
What are the key details on Limitations and What Developers Should Do?
The base model is not suitable for new robots out of the box: it must be fine-tuned. Adapted versions already exist for popular platforms. The model can be downloaded from Hugging Face after registration and acceptance of the access terms, and the code is on GitHub. The requirements are substantial: the standard version needs a graphics card with 80 GB of memory.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.