SkeLAM teaches a robot to mimic human movements after training on 10 hours of data

About 10 hours of robot recordings and one human-hand demonstration. That was enough for researchers at Harbin Institute of Technology (Shenzhen campus) to build SkeLAM, a model that transfers motion from a human to a robot arm without fine-tuning. The system is built on the pretrained video model Cosmos-Predict2.5-2B, which the team trained on roughly 10 hours of robot data.

Human in, robot arm out

The experiment is simple. The model was trained on data in which a robot arm interacts with objects. At test time, the researchers replaced the robot frames with a human demonstration: a live hand moves in the frame. The generated video shows a mechanical arm whose movement matches the human gesture. The direction of motion is the same as in the demonstration.

The important detail is how this works. The model's parameters are fully frozen during the transfer, and nothing was fine-tuned on the human demonstrations. This mode is called zero-shot: the model meets data of a different nature and handles it with the weights it learned from robot data.

Why transfer between bodies remains a bottleneck

Robotics has a long-standing data shortage. Teleoperation recordings are expensive: they require an operator, equipment and time. Videos of people moving their hands are many times easier to shoot, and there are huge numbers of them online. But a human hand and a robot gripper differ in geometry, degrees of freedom and appearance. A model trained on one "body shape" usually reads another one poorly. In the literature, this problem is called cross-embodiment transfer.

SkeLAM shows that a small amount of robot data can be enough to build a bridge between two embodiments. By industry standards, ten hours is a modest dataset. Part of the work is done by the large video model, which already has a sense of scene physics, objects and motion dynamics.

Experiment parameters

• Authors: Harbin Institute of Technology, Shenzhen campus
• Model name: SkeLAM
• Base model: Cosmos-Predict2.5-2B, a pretrained video model with about 2B parameters
• Training data: about 10 hours of robot interaction data in total
• Test scenario: transfer from a human hand to a robot arm
• Test input: a human video demonstration instead of robot frames
• Result: the mechanical arm in the generated video moves in a direction that matches the demonstration
• Transfer mode: all parameters frozen, no fine-tuning on these demonstrations

What remains to be verified

Generating video and controlling a real robot arm are different steps. A clip in which the arm moves in the right direction does not yet guarantee that a physical robot will perform a grasp, a rotation or precise positioning. Matching the direction of motion tests the basics, but not fine motor control. Further work is likely to focus on converting generated trajectories into commands for real robots and testing them on different platforms.

For people building robots, the practical takeaway is this: if further tests confirm the result, a significant share of training data could be collected not on a robot but from people with an ordinary camera. That lowers dataset costs and speeds up iteration. Cosmos-class video models are turning from clip generators into a base for control policies. It is worth watching how the team carries the result over to physical platforms, and what task success metrics it reports outside demo videos.