Tsinghua Team Releases HandsOn: 66 Tasks to Test Robot Dexterity

Researchers at Tsinghua University have published HandsOn, a benchmark for evaluating dexterous robot manipulation. It includes 66 tasks that test how well a machine handles two multi-fingered hands at once, from grasping and moving objects to using tools and making delicate contact. The authors start from a simple premise: it is time to rewrite the "exam questions" for robot hands, because the old ones were written for different hardware.

Why the old tests stopped working

For decades, most popular manipulation benchmarks were built around parallel-jaw grippers, meaning two "pads" that squeeze an object. For that kind of mechanism the task set is straightforward: pick up, move, insert, press. There is little to evaluate beyond positioning accuracy and action sequencing.

A multi-fingered hand is different. It has dozens of degrees of freedom, its fingers can move independently, and the same object can be held in many ways. A test that boils down to "grab it and put it down" does not show whether a robot can reposition an object in its palm, use a screwdriver, or gently steady with one hand whatever the other hand is working on. HandsOn fills that gap and puts bimanual work at the center.

Ego data as fuel for dexterous hands

The second reason for the benchmark is data. Recordings of human actions from a first-person view keep piling up: cameras on heads or glasses capture how people pick up things, use tools, and perform precise contact actions. Such datasets show what is hard to collect any other way, namely natural fine motor skills in everyday and work settings.

This data is of little use for parallel grippers, because human hand movements have to be heavily simplified. A dexterous hand is structurally similar to a human one, so "how a human does it" annotations can be transferred to the robot with far smaller losses. Hence the authors' logic: if the industry plans to train robots on ego video, it needs a shared way to measure results, or approaches simply cannot be compared.

What we know about HandsOn:

• 66 tasks in a single set;
• a focus on dexterous multi-fingered hands rather than parallel grippers;
• bimanual scenarios as a separate object of evaluation;
• action categories: grasping, tool use, fine contact;
• developer: a research group at Tsinghua University.

Who will find this useful in practice

The first users of the benchmark will be teams building control policies for humanoids and for manipulators with dexterous hands. A single task set lets them compare models trained on human video against approaches based on teleoperation and simulation, without inventing new scenes each time. For hardware makers, it is a chance to see where the design of the fingers or tactile sensors becomes a bottleneck.

There is also a more down-to-earth effect. As long as dexterous hands have no generally accepted test, any claim of "human-level dexterity" remains a demo on a few lucky clips. A standardized set of 66 tasks raises the cost of such claims: a result either reproduces on common tasks or it does not. For investors and customers choosing between platforms, this is one of the few ways to tell real progress from a well-edited video.

HandsOn's success will depend on whether other labs adopt it. If the benchmark catches on, it may become for dexterous hands what earlier standard task sets became for gripper-based manipulation: a common language for comparing models.