Transformers at the wheel: how Yandex uses AI to control self-driving vehicles

Yandex Autonomous Transport has revealed technical details on the application of transformer architectures in Motion Planning — the component of an autonomous vehicle that makes real-time decisions about trajectory. The path from lab experiments to real-world road testing proved to be more complex than it appears from the outside.

From Language to Road

The transformer architecture has achieved dominant status in NLP and computer vision thanks to the attention mechanism: the model can simultaneously account for context from different parts of input data. In the task of controlling a vehicle, this is equally valuable — the system must process the position of dozens of objects around the car within fractions of a second, predict their behavior, and select a safe maneuver. However, transferring architecture from NLP to autopilot is not a trivial task.

In a language model, an inaccurate word is a typo in the text that a reader will forgive. In an autonomous vehicle, deviation from an optimal trajectory at a critical moment is a potential accident. This fundamentally changes the model requirements: from quality metrics to the entire testing culture.

Open Loop vs Closed Loop

In the autonomous driving industry, there are two fundamentally different approaches to system evaluation:

• Open Loop — the model predicts trajectory on historical data, the result is compared against actual driver behavior. Fast, cheap, allows iteration at large scale.
• Closed Loop — the system itself controls the vehicle — real or in simulation — and is evaluated by what happens in dynamics: did the car avoid collisions, how comfortable was the ride, how did it handle unusual situations.

Both modes are needed, but they provide fundamentally different information about system reliability. Yandex places special emphasis on Closed Loop testing despite its higher cost and slower pace. The logic is: in real driving, errors are not isolated — a deviation at time T affects situation perception at T+1, and errors compound. Open Loop does not see this chain and can give a false sense of safety.

Why Good Metrics Are Deceptive

The main counterintuitive finding: high trajectory prediction accuracy — low L2 error — correlates poorly with real driving safety. A model can demonstrate excellent results in tests, accurately reproducing average driver behavior on historical data, while driving worse than a simpler but more robust system. The reason lies in the nature of Closed Loop: each system decision affects the next situation. Open Loop metrics measure each step independently and fail to capture this dynamic.

"Safety is more important to us than any architecture," —

Maksim, head of behavior and motion prediction at Yandex Autonomous Transport.

This is why the team builds a multi-level verification system: from formal metrics on data — through simulation — to private and public tests on real vehicles. Model architecture is just one parameter, not the foundation of a safety system.

Path from Lab to City

The Yandex team has traveled from early ML experiments, when neural network approaches were tested in parallel with classical planning algorithms, to regular autonomous vehicle tests in real urban traffic. At each stage, the transformer architecture adapted to strict requirements: predictability of behavior in typical situations, robustness to rare scenarios, real-time operation with limited computational resources on board. Motion Planning is not simply predicting the next point. It is decision-making under uncertainty every second, with responsibility for human lives.

What This Means

Transformers are coming to autonomous transport — but not as a ready-made solution exported from the world of language models. Every new application requires rethinking metrics, test infrastructure, and safety culture. For the entire industry, this is an important signal: success in NLP does not automatically transfer to domains where the cost of an error is measured not by model rating, but by human lives.