Waymo and tornadoes: how Google DeepMind teaches autonomous vehicles to survive in chaos

Imagine: an empty highway somewhere in Oklahoma, sunset, and suddenly a giant tornado descends from the clouds right in the vehicle's path. What should an autonomous vehicle do in such a situation? Previously, engineers would have had to either wait for years for a real disaster to occur, hoping their test car would be nearby and survive, or try to manually program wind physics and debris in an old game engine. But Waymo decided to take a different path, joining forces with the geniuses from Google DeepMind. Together they created a system based on Genie 3 — a neural network that transforms text prompts into interactive and eerily realistic 3D worlds.

Actually, Genie was originally conceived as a fun toy for generating simple platformers in the spirit of Super Mario. But at DeepMind, they quickly understood that if the model can grasp the physics of a pixel character's jump, it can just as easily handle the dynamics of a multi-ton truck on an icy highway. For Waymo, this is not just another graphics upgrade in a simulator. It's a transition to full-fledged "world models" (World Models) that are adapted to the harsh demands of real road traffic. Now developers can literally write any kind of chaos into a scenario, from an elephant running onto the highway to a dust storm, and see how the AI behaves.

The problem is that training an autonomous vehicle to drive through sunny San Francisco or Phoenix today is not that difficult. The real challenge for the industry is the so-called "edge cases." These are the very events that happen once in a million miles, but they are precisely the ones that cause fatal accidents. It's physically impossible to collect data on such incidents on real roads — too expensive and simply dangerous for others around. Thanks to Genie 3, Waymo can generate thousands of variations of the same dangerous scenario at the snap of a finger, forcing algorithms to learn from their mistakes in virtuality.

An important difference between Genie 3 and classical simulators is that this neural network creates an environment that reacts to the actions of the agent. If a car in the simulation suddenly turns the wheel or hits the brake, the world around it changes in accordance with the physics predicted by the neural network. This is an attempt to teach artificial intelligence to understand the cause-and-effect relationships of our reality, rather than simply copying the behavior of human drivers according to a template. This is a fundamental shift in how we approach the safety of autonomous systems.

This move looks like a direct response to skeptics and competitors, including Tesla. While Elon Musk bets on mass data collection from millions of ordinary trips by his clients, Waymo is moving into deep theory and extreme simulation. This is a classic clash of approaches: brute force against intelligent preparation in the laboratory. If Waymo can prove that their "digital twins" of chaos adequately reflect reality, regulators could become much more lenient about the mass launch of driverless robotaxis.

Of course, the question remains open: how accurately do the hallucinations of Genie 3's neural network correspond to the real physics of a tornado or the behavior of a frightened animal? If the model miscalculates tire grip on the road during rain, training in such a simulation could even be harmful, instilling false skills in the algorithm. But given DeepMind's expertise in deep learning, this is perhaps the most serious attempt to create a full-fledged testing ground for AI that we have seen in recent years.

The bottom line: Waymo is shifting the arms race from the real world to virtual simulations. Will other companies be able to create such high-quality "world models," or will autonomous vehicle safety become a monopoly of those with access to Google's computing power?