AI that learns how the physical world works is coming to robots and cars

A new kind of AI is moving from labs into products. It is called a world model, and it learns how the physical world behaves by studying images and video. Wired reports that 2027 could be a big year for home robots and driverless cars, both helped by this technology. Most people will never use a world model directly. They may still feel its effects.

What a world model is

Chatbots work with text. Given a sentence, they predict the next word. A world model works with pictures. Oliver Cameron, CEO of the AI lab Odyssey, says you give it an image of the world and it predicts the next visual state of the world. The result is a 3D scene that follows the rules of physics. In his words, these are simulations that look and feel real.

How robots and cars use it

The main use is practice. Brian McClendon, chief technology officer of Niantic Spatial, says someone could photograph the place a robot will work, build a 3D model of it, and run millions of simulations. The robot learns the space without moving a muscle.

Cars follow a similar path. Wayve builds world models for driving, partly from dashcam footage. It uses them to recreate dangerous roads and unsafe conditions that would not be practical to test for real. Jamie Shotton, Wayve's chief scientist, says a world model must understand cause and effect, including how people and objects react to the car's own actions.

Who is putting money in

• Niantic sold Pokémon Go last year and now builds world models, drawing on its trove of real-world scans.
• AMD acquired Fei-Fei Li's startup World Labs this year for over $8 billion.
• Yann LeCun, Meta's former chief AI scientist, raised over a billion dollars for his own startup.
• Tencent is investing in world models for video games, and Google keeps touting its progress.

Where your data comes in

These systems need real footage of real life. Wired writer Reece Rogers strapped an iPhone to his forehead and recorded hours of himself folding laundry, washing dishes and pouring soda. He sold the video through an app for what he calls a fairly measly payout. The article says more companies building world models could create more gig work that asks for this kind of recording.

What is not clear yet

The article gives no launch dates for specific home robots or driverless services. It says only that 2027 could be a major year. It also gives no figures on what data collection jobs pay beyond calling one payout measly. Anyone thinking of selling home video should read what an app does with the footage first.