Robots Can't Lie Anymore: Why 'Success Rate' No Longer Matters

Imagine you hire a driver and he says: "I reach the destination in 90% of cases." In robotics, this has long been considered an excellent result. We've gotten used to measuring progress through a success coefficient, ignoring exactly how that success was achieved. A robot could perform a series of absurd, energy-intensive movements, nearly breaking its manipulator, but ultimately touch the right button — and voilà, the cherished one gets written in the report. Chinese researchers decided it was time to stop this imitation of feverish activity and presented a new paradigm for evaluating embodied intelligence. This is not just a cosmetic change in metrics, but a fundamental shift in how we understand "intelligent" machines.

The problem with the old approach was its binary nature. Either victory or defeat. But in the real world, beyond pristine laboratories, the cost of that victory matters. The new evaluation methodology — embodied manipulation — introduces a multidimensional scale. Now it's not just the end result that's counted, but the trajectory of movement, the time spent, and most importantly, resilience to external interference. If you give the robot a slight push or change the lighting, and its "success rate" drops from 90% to zero, then there was no intelligence there. There was only a rigid program optimized for a specific investor video.

Why is this needed right now? We're on the verge of mass deployment of humanoids and manipulators into unstructured environments — our homes and offices. There are no ideal conditions here. There are children, pets, and ever-changing chaos. The old metric is useless when it comes to safety and predictability.

The new paradigm forces developers to focus on generalization, not memorization of specific scenarios. It's a harsh filter that quickly weeds out startups trading in pretty renders from companies creating real technology.

The transition to complex metrics also changes the rules of the game in model training. When a neural network receives a reward not just for "achieving a goal," but for "efficiently and safely achieving a goal," its behavior changes. It becomes more like a living creature, conserving energy and avoiding unnecessary risks. That's what we call true embodied intelligence.

Researchers emphasize that abandoning the dictatorship of a single metric will finally allow the industry to compare different approaches objectively. Previously, each laboratory boasted its own numbers that couldn't be compared. Now a unified scale is emerging, and it's quite harsh.

For the market, this means a temporary slowdown in "paper" successes, but a sharp acceleration of real progress. We'll see fewer headlines about "robots that do everything" and more boring but important graphs about robustness and control quality. This is the industry growing up.

You can no longer just record the hundredth take where the robot happened to handle the task and pass it off as a breakthrough. Now the system must prove its effectiveness in dynamics, under load, and in conditions of uncertainty.

The key point: The era of marketing videos without editing cuts is coming to an end. Now robots will have to prove their fitness for duty with numbers that can't be faked by simple luck. Are today's market leaders ready for such a level of transparency?