Qwen3.5-4B after CaPtain RL fine-tuning solves 73.8% of LIBERO tasks, versus 0% for the base model

The MMLab group at the Chinese University of Hong Kong, together with NovaXBot, has released CaPtain, an open post-training pipeline that turns a small open vision-language model into a code-as-policy agent. The agent looks at camera images, writes Python code on top of perception and motion APIs, and reads back what physically happened. In this setup, the base Qwen3.5-4B does not solve a single LIBERO episode. After SFT and multi-turn RL, the resulting CaPtain-4B reaches 73.8% on the held-out split of this benchmark. The authors have published the code, data and weights.

Why small models haven't controlled robots with code until now

Frontier models with a code harness already handle manipulation well. According to the paper, GPT-6 Astra solves 94.0% of LIBERO-PRO episodes in this mode (LIBERO-PRO is a version of LIBERO with altered objects, layouts and instructions). But the approach has a price: OpenRUA, for example, scores 91.8 on LIBERO-PRO at a cost of $8–22 per episode. Such an agent cannot be fine-tuned for a new robot and cannot run on an onboard computer.

The workarounds are limited too. Some systems hand manipulation off to a frozen VLA model or to ready-made skills such as grasp and place. In those designs the code only chooses which skill to call, while the skill itself decides where and how to grasp. At best, RL in such a scheme learns to sequence calls.

A 22-primitive harness and a critic with access to the physics

CaPtain-Harness gives the agent only low-level functions: 22 primitives for perception, geometric measurement and guarded motion. SAM 3 supplies masks and points, and motion commands accept an explicit end-effector pose or a gripper command. There are no ready-made skills and no VLA, so the policy itself chooses the approach direction and when to close the gripper. Every call returns a text report: measured poses, contact force, and the reason a motion stopped.

An episode is up to 60 queries. The history is truncated: the context keeps the 12 most recent images and no more than 20 report lines per cell. When the history is compressed, the Python session variables are preserved, so discovered poses and helper functions are not lost. The same logic applies in both training and testing.

Training has two stages. First comes SFT on trajectories generated by GPT-6 Astra in the same multi-turn format. Then comes multi-turn RL.

The main difficulty here is credit assignment: the outcome is known only at the end of an episode, while the effect of a grasp shows up several turns later. The authors added a physics critic. It reads the simulator's privileged state (exact object poses, grasp status, subgoal progress), which the policy cannot see.

The critic assigns credit to each query based on how much that query changed the predicted probability of success. Queries that only look or verify get almost zero, while credit concentrates on grasps and placement. At test time the critic is switched off, and the policy relies only on the cameras and reports.

Results and limits of applicability

• LIBERO: 0% for the base model, 55.4% after SFT, 73.8% after RL.
• LIBERO-PRO: 78.5%, higher than several agents built on frontier models. The RL gain across the three perturbation sets ranges from 17.6 to 21.2 points.
• RL added 19.4 points to the SFT model across the four LIBERO-PRO sets. For AgenticRoboRL, where RL runs on top of a frozen VLA, the gain is 5.9 points, although the starting baselines of the two systems differ.
• SimplerEnv WidowX: 83.3% without training on the target domain.
• CaP-Bench: 75.0 points, higher than frontier models and a 7B coder trained with RL on this benchmark.
• On a real robot, the model performs tasks after adaptation in only a rough simulation twin.

For comparison, the authors cite end-to-end policies. One of them drops from 97% on standard LIBERO to 12.8% on LIBERO-PRO. RL post-training of RIPT-VLA on LIBERO-Plus yields 68.4 versus 69.6 for the base OpenVLA-OFT. With CaPtain, all seven perturbation dimensions of LIBERO-Plus improved.

The authors name the limitations themselves. On RoboTwin 2.0, which requires coordinating two arms, success is only 22.6%. On RoboDojo, with unfamiliar arm kinematics, it is 0.8%. The critic works only in simulation, so the method is not suitable for tasks without a good simulation model.

The practical takeaway for roboticists: manipulation through low-level primitives can be fine-tuned with the same recipe of SFT plus multi-turn RL used for coding agents. The result is a 4B policy that costs a small fraction of what frontier agents do and runs without a VLA at test time. Open weights and data are already available for testing on other robots.