HumanoidPose Estimates Humanoid Robot Pose From Video, Cutting Joint-Angle Error by 18–32%

Researchers have introduced HumanoidPose, a model that recovers a humanoid robot's full joint configuration in world coordinates from ordinary video. On three real humanoid platforms, it cuts joint-angle error by 18–32% relative to the best baseline method for each platform. Most of the training data is synthetic: human motions were retargeted onto 16 different humanoid designs.

Vision instead of sensors

The authors call the task "vision as proprioception." A robot normally knows where its joints are thanks to built-in sensors. But when it interacts with someone else's machine, that data is often unavailable because of privacy, ownership, or communication-link limits. There is a second problem. Videos of robots online, along with AI-generated clips, exist in huge volumes, but they come without joint angles. That makes them hard to use directly for robot training.

HumanoidPose addresses both gaps: it needs only a video sequence, and it outputs the pose of the whole body.

How the model is built

The model combines two sources. Pretrained features for human pose estimation are merged with robot kinematics in a single model that predicts the joint configuration for each specific robot design. To build the training data, the authors retargeted human motions onto 16 humanoids. Labels come for free, because the joint angles are known by construction.

Synthetic data usually transfers poorly to real footage. So the team added self-training: unlabeled online videos are used to create a 2D robot pose detector. Its purpose is robustness on real recordings, where lighting, camera angles, and backgrounds differ from the simulation.

What the measurements show

• Joint-angle and joint-position accuracy is higher than that of all compared baseline methods.
• Joint-angle error is 18–32% lower than the best baseline method on each of the three real humanoid platforms.
• Global localization, meaning the robot's position in world coordinates, is improved.
• The model generalizes better to designs that were not in the training set.

The limitations are visible too. The description does not make clear whether the system runs in real time, how robust it is to occlusion, or which platforms were in the test. The figures come from a set of baselines chosen by the authors, and there is no independent verification yet.

Where this could be useful

The first scenario: a robot needs to understand what another machine nearby is doing, but has no access to its telemetry. A camera is enough to estimate the pose of a partner or a competitor in the shared workspace.

The second scenario matters more for developers. If joint angles can be extracted from video, online clips and video-model generations become a source of trajectories for policy training. Expensive data collection on real hardware could be partly replaced by video archives, provided the pose estimation quality holds up on your platforms. Teams working with humanoids should watch whether the authors release code and weights: that determines whether HumanoidPose becomes a tool or stays a paper.