Robots learn from 20 854 hours of video: how Sharpa is building dexterous manipulation

Data for robots is getting more expensive faster than models are growing, and the industry is looking for a way out. The EgoScale team, which includes NVIDIA researchers, pretrained a robot policy on 20 854 hours of first-person video of people performing tasks. They recorded a log-linear relationship: more hours give better results, and the growth is predictable. Against this backdrop, the Chinese outlet Jiqizhixin examines Sharpa's approach, which treats dexterous manipulation as a single system with three layers: hardware, data and model.

An important caveat: the available excerpt of the source material contains no Sharpa benchmarks of its own. What follows is therefore an analysis of the context and logic of the approach, not of its verified numbers.

Why data is moving away from teleoperation

Just a year ago, teleoperation was the main source of training trajectories for manipulators. An operator in a VR headset or wearing a glove controls the robot while the system records the actions. The method yields clean data in the robot's "native" format, but it has three weak points. It is expensive, because it requires an operator, a fleet of robots and a site. It is slow: an hour of recording takes exactly an hour of real time. And it scales poorly across geography, since every new scene requires physical presence.

Video of humans works differently. A person with a camera on their head can film home life, cooking, assembly and any fine motor activity with no robot nearby. That explains the scale of EgoScale. Works like HumanScale, according to Jiqizhixin's announcement, go further and argue that video can become the primary data source.

Where hardware comes in

Video has a price: the gap between the human body and the robot body. The human hand has dozens of degrees of freedom and tactile sensation. A two-finger gripper cannot replicate what the video shows. The closer the kinematics of the robot's hand are to the human hand, the less is lost when transferring motion from a clip into a control policy.

This is why dexterous manipulation does not split cleanly into separate tasks of "build the hand" and "train the model." The choice of hand design determines which data can be used. The data format determines the model architecture. The model, in turn, places demands on the sensors. The headline of the Jiqizhixin piece says that Sharpa is trying to work through this whole chain rather than optimize a single link.

Key takeaways

• EgoScale pretraining volume: 20 854 hours of egocentric human video.
• Pattern: a log-linear curve between data volume and policy quality.
• Bottleneck of teleoperation: cost, recording speed and difficulty of scaling.
• Sharpa's approach: a "hardware – data – model" chain for dexterous manipulation.
• Open question: the excerpt contains no comparable public Sharpa metrics.

What to watch next

A log-linear curve implies diminishing returns: each doubling of data gives roughly the same gain. For developers, this is a practical signal that helps plan the data budget in advance. For teams building their own hands and manipulators, the conclusion is different: kinematic compatibility with the human hand becomes a competitive advantage, because it opens access to huge video collections.

When evaluating such systems, it is worth checking three things. First: which tasks success is measured on, and whether they include long-horizon operations with small objects. Second: what share of the data comes from video and what share from teleoperation for fine-tuning. Third: whether the results reproduce on other platforms, not only on the authors' "native" hand. Until there are public answers to these questions, claims of a systemic approach remain a claim, not a proven result.