On the road with AI
The Waymo World model isn’t just a straight port of the Genie 3 with dashcam videos stuffed inside. Waymo and DeepMind used a special post-training process to train the new model to generate 2D video and 3D lidar output of the same scene. While cameras are great for seeing fine details, Waymo says lidar is necessary to add significant depth information to what a self-driving car “sees” on the road — perhaps someone should tell Tesla about that.
Using the world model allows Waymo to take video from its vehicles and use signals to change the vehicle’s path, using what it calls Driving Action Control. These simulations, which come with lidar maps, reportedly provide greater realism and consistency than older reconstruction simulation methods.
With the world model, Waymo can see what would happen if the car took a different turn.
This model could help improve self-driving AI without even adding or removing everything. There are plenty of dashcam videos available for training self-driving vehicles, but they lack the multimodal sensor data of Waymo’s vehicles. Feeding such video into the Waymo World model generates matching sensor data, which shows how the driving AI would have seen that situation.
While the Waymo World model can create completely synthetic scenes, the company is more interested in “converting” the situations in the real video. Blog posts include examples of changing the time of day or season, adding new signage, or placing vehicles in unusual locations. Or, hey, why no elephants on the road?
If an elephant comes, Waymo is ready.
Waymo’s early test cities had consistently sunny weather (like Phoenix) with little inclement weather. This type of simulation can help cars adapt to more diverse conditions. New markets include places with more difficult conditions, including Boston and Washington, D.C.
Of course, the benefit of the new AI model will depend on how accurately Genie 3 can simulate the real world. The test videos we’ve seen of the Genie 3 run the gamut from pretty reliable to uncanny valley territory, but Waymo believes the technology has improved to the point that it could teach self-driving cars a thing or two.
<a href