Google’s new AI model creates video game worlds in real time

Google DeepMind is releasing a new version of its AI “world” model, called Genie 3, capable of generating 3D environments that users and AI agents can interact with in real time. The company is also promising that users will be able to interact with the worlds for much longer than before and that the model will actually remember where things are when you look away from them.

World models are a type of AI system that can simulate environments for purposes like education, entertainment, or to help train robots or AI agents. With world models, you give them a prompt and they generate a space that you can move around in like you would in a video game, but instead of the world being handcrafted with 3D assets, it’s all being generated with AI. It’s an area Google is putting a lot of effort into; the company showed off Genie 2 in December, which could create interactive worlds based off of an image, and it’s building a world models team led by a former co-lead of OpenAI’s Sora video generation tool.

But the models currently have a lot of drawbacks. Genie 2 worlds were only playable up to a minute, for example. I recently tried “interactive video” from a company backed by Pixar’s cofounder, and it felt like walking through a blurry version of Google Street View where things morphed and changed in ways that I didn’t expect as I looked around.

Genie 3 seems like it could be a notable step forward. Users will be able to generate worlds with a prompt that supports a “few” minutes of continuous interaction, which is up from the 10–20 seconds of interaction possible with Genie 2, according to a blog post. Google says that Genie 3 can keep spaces in visual memory for about a minute, meaning that if you turn away from something in a world and then turn back to it, things like paint on a wall or writing on a chalkboard will be in the same place. The worlds will also have a 720p resolution and run at 24fps.

DeepMind is adding what it calls “promptable world events” into Genie 3, too. Using a prompt, you’ll be able to do things like change weather conditions in a world or add new characters. 

However, this probably isn’t a model you’ll be able to try for yourself. It’s launching as “a limited research preview” that will be available to “a small cohort of academics and creators” so its developers can better understand the risks and how to appropriately mitigate them, according to Google. There are also plenty of restrictions, like the limited ways users can interact with generated worlds and that legible text is “often only generated when provided in the input world description.” Google says it’s “exploring” how to bring Genie 3 to “additional testers” down the line.