Open-source Physical AI pipeline for the SO-101 arm built on ROS2 and LeRobot for 30,000 rubles

The open-source community now has a practical Physical AI stack for the low-cost SO-101 manipulator. It covers the entire path from a teleoperated demonstration to autonomous task execution on a real robot. Instead of a pile of disconnected scripts, the project ties ROS2, LeRobot, and imitation learning into a single reproducible pipeline for a setup that costs roughly 30,000 rubles.

How the stack works

The project's main idea isn't a new model. It's that robotics and ML no longer live in separate worlds. The SO-101 manipulator sits at the bottom. Above it is a ros2_control layer with a hardware interface for the Feetech STS3215 servos, and on top of that come teleoperation, cameras, episode recording, and inference. As a result, the system sees the robot as a proper ROS2 device rather than a collection of scripts tied to a single board. That makes the stack portable and easy to extend.

Leader/follower teleoperation runs on top of this layer: the operator demonstrates the desired motion, and the follower arm repeats it while generating training data. During demonstrations, the project records episodes to rosbag or MCAP, supports multiple cameras, and lets you inspect observations and actions through visualization in Rerun. This matters: you can not only collect data but also quickly discard bad episodes before training if the synchronization, camera angles, or trajectories turn out poorly.

The path from data

After recording, the project converts episodes from ROS formats into a LeRobot dataset. This bridges the ROS2 and ML worlds, eliminates homegrown intermediate formats, and gets you to policy training faster. From there, you can try end-to-end imitation learning with models such as ACT or SmolVLA and then deploy the resulting policy back into the robot's ROS2 control loop. Another advantage of this approach is that it builds on an existing tool ecosystem.

In practice, the full workflow looks like this:

• bring up the manipulator and launch ros2_control
• collect demonstrations via leader/follower teleop
• record episodes to rosbag or MCAP
• inspect camera streams, actions, and observations in Rerun
• convert to a LeRobot dataset, train a policy, and run it on the robot

Another strength of the stack is that it separates the robot-side runtime from the heavy model. If the local computer next to the arm isn't powerful enough, the policy can run on an external GPU server through policy_server, while the robot side keeps only the inference client and the execution loop. For Physical AI, this isn't cosmetic. It's sound engineering decoupling: the control loop stays close to the hardware, while the "brain" scales independently. This makes it easier to experiment with heavier models and reduces the hardware requirements on the robot side.

Where the practical value lies

Projects like this usually break at the boundaries between disciplines: the robot can move, but the data is collected badly; the dataset exists, but getting it back onto the hardware is painful; the model trains, but doesn't survive in a real runtime. This project covers exactly the most tedious but most valuable parts: bringup, recording, visual inspection, conversion, and integration back into ROS2. That's why the stack looks less like a research demo and more like a hands-on learning and applied platform for embodied AI.

The project is especially useful for anyone who wants to get into Physical AI without a lab budget. On the inexpensive SO-101, you can first debug the basics—power, calibration, teleop, cameras, data schema, and inference latency—and only then move on to more complex manipulators. This approach saves months: you build a reproducible pipeline first and experiment with policies second, not the other way around. For students, research teams, and small startups, it noticeably lowers the barrier to entry.

What it means

Physical AI is gradually moving out of presentation mode and into the era of reproducible open-source stacks. If you can bring up an inexpensive manipulator as a ROS2 system, collect demonstrations on it, train a policy, and deploy it back to the hardware, the barrier to entry for embodied AI drops significantly for engineers and small teams alike. That means more experiments will happen on real workbenches and in labs rather than on slides.