Transformers at the Wheel: How Yandex Uses AI to Drive Its Self-Driving Cars

Yandex Autonomous Transport has revealed technical details of how it applies transformer architectures to Motion Planning, the component of a self-driving car that decides on the vehicle's trajectory in real time. The path from lab experiments to tests on real roads turned out to be harder than it looks from the outside.

From language to the road

The transformer architecture came to dominate NLP and computer vision thanks to the attention mechanism: the model can weigh context from different parts of the input at once. That is valuable for driving, too: within fractions of a second, the system has to process the positions of dozens of objects around the car, predict their behavior and choose a safe maneuver.

But porting the architecture from NLP to an autopilot is not a trivial task. In a language model, an inaccurate word is a slip in the text that a reader will forgive. In a self-driving car, a deviation from the optimal trajectory at a critical moment is a potential crash. This fundamentally changes the requirements for the model, from quality metrics to the entire testing culture.

Open Loop versus Closed Loop

The self-driving industry has two fundamentally different approaches to evaluating a system:

• Open Loop: the model predicts a trajectory on historical data, and the result is compared with the actual behavior of a human driver. It is fast, cheap and allows iteration at large scale.
• Closed Loop: the system drives the car itself, either a real one or in simulation, and is judged by what happens dynamically: whether the car avoided collisions, how comfortable the ride was, and how it handled non-standard situations.
• Both modes are needed, but they provide fundamentally different information about the system's reliability.

Yandex puts particular emphasis on Closed Loop testing, even though it is more expensive and slower. The logic is this: in real driving, errors are not isolated. A deviation at moment T affects how the situation is perceived at T+1, and errors build up. Open Loop does not see this chain and can give a false sense of safety.

Why good metrics mislead

The main counterintuitive finding: high trajectory prediction accuracy (a low L2 error) correlates poorly with the safety of real driving. A model can post excellent test results by accurately reproducing the behavior of the average driver on historical data, and still drive worse than a simpler but more robust system.

The reason lies in the nature of Closed Loop: every decision the system makes affects the next situation. Open Loop metrics measure each step independently and do not capture this dynamic.

"For us, safety matters more than any architecture," said

Maxim, head of the behavior and motion prediction team at Yandex Autonomous Transport.

That is why the team builds a multi-level verification system: from formal metrics on data, through simulation, to private and public tests on real vehicles. The model architecture is just one parameter, not the foundation of the safety system.

From lab to city

The Yandex team has gone from its first ML experiments, when neural-network approaches were tested alongside classical planning algorithms, to regular tests of self-driving cars in real urban traffic. At every stage, the transformer architecture was adapted to strict requirements: predictable behavior in typical situations, robustness in rare scenarios, and real-time operation with limited onboard computing resources.

Motion Planning is not simply predicting the next point. It is making decisions under uncertainty every second, with responsibility for human lives.

What this means

Transformers are arriving in autonomous transport, but not as a ready-made solution exported from the world of language models. Every new application requires rethinking metrics, test infrastructure and safety culture. For the whole industry, this is an important signal: success in NLP does not transfer automatically to a field where the cost of an error is measured not in model ranking but in human lives.