SkeLAM taught a robot to mirror human movements after training on just 10 hours of data

Roughly 10 hours of robot recordings and a single human-hand demonstration. That was enough for researchers at Harbin Institute of Technology (Shenzhen campus) to build SkeLAM, a model that transfers motion from a human to a manipulator with no fine-tuning. The system is built on a pretrained video model, Cosmos-Predict2.5-2B, and was trained on roughly 10 hours of robot data.

Human in, manipulator out

The experiment is set up simply. The model was trained on data in which a robotic arm interacts with objects. At test time, the robot footage was swapped for a human demonstration: a live hand moves in the frame. The generated video shows a mechanical arm whose movement matches the human gesture. The direction of motion matches the demonstration.

The key detail is how this was achieved. The model's parameters were kept fully frozen during the transfer — nothing was fine-tuned on the human demonstrations. This mode is called zero-shot: the model encounters data of a different nature and handles it using only the weights it learned from robot data.

Why cross-embodiment transfer remains a bottleneck

Robotics has a longstanding data-scarcity problem. Teleoperation recordings are expensive: they require an operator, equipment, and time. Video of a person moving their hands is far easier to capture, and there's an enormous amount of it online. But a human hand and a manipulator's gripper differ in geometry, degrees of freedom, and appearance. A model trained on one "body shape" usually struggles to read another. This problem is known in the literature as cross-embodiment transfer.

SkeLAM shows that a small amount of robot data can be enough to build a bridge between two body forms. Ten hours is a modest dataset by industry standards. Part of the heavy lifting is handled by the large video model, which already carries a representation of scene physics, objects, and movement dynamics.

Experiment parameters

• Authors: Harbin Institute of Technology, Shenzhen campus
• Model name: SkeLAM
• Base model: Cosmos-Predict2.5-2B, a pretrained video model with roughly 2 billion parameters
• Training data: a total of roughly 10 hours of robot interaction data
• Test scenario: transfer from a human hand to a robotic arm
• Test input: a human video demonstration in place of robot footage
• Result: the mechanical arm in the generated video moves in a direction matching the demonstration
• Transfer mode: all parameters frozen, no fine-tuning on these demonstrations

What still needs to be verified

Video generation and controlling a real manipulator sit on different rungs of the ladder. A clip where the arm moves in the right direction doesn't yet guarantee that a physical robot will execute a grasp, a turn, or precise positioning. Matching movement direction validates the foundation, but not fine motor control. Further work will likely move toward converting generated trajectories into commands for real robots and testing across different platforms.

For those building robots, the practical takeaway is this: if further checks confirm the result, a substantial share of training data could be collected not from a robot but from people filmed with an ordinary camera. That lowers the cost of datasets and speeds up iteration. Video models in the Cosmos class are becoming more than just clip generators — they're turning into a foundation for control policies. Worth watching is how the team carries this result over to physical platforms and what task-success metrics it shows outside of demonstration videos.