Google DeepMind debuts Genie 3 model for generating interactive virtual worlds

Alphabet Inc.’s Google DeepMind lab today detailed Genie 3, an artificial intelligence model designed to generate virtual worlds.

The AI is the third iteration of an algorithm series that the company introduced last February. According to DeepMind, it could help advance machine learning research by creating higher-quality training environments for AI models.

Genie 3 generates virtual worlds based on natural language prompts. It can simulate forecasts, Alpine landscapes and other outdoor locations along with indoor spaces. In one internal test, DeepMind researchers had Genie 3 generate an elaborate underwater environment complete with a large jellyfish.

Users can change a virtual world with prompts. Genie 3 may be instructed to modify the weather, adjust the camera angle or add objects to an environment. It’s also possible to simulate interactions between those objects. 

The algorithm’s predecessor, Genie 2, could render virtual environments for up to 20 seconds at a time. DeepMind says that Genie 3 can manage up to several minutes. The Alphabet unit’s researchers also boosted rendering quality from 360p to 720p, which corresponds to a resolution of 1280 by 720 pixels.

Rendering consistency is another area where Genie 3 offers improvements. During user sessions, the model analyzes past frames to determine how future frames should be generated. 

“Generating an environment auto-regressively is generally a harder technical problem than generating an entire video, since inaccuracies tend to accumulate over time,” DeepMind researchers Jack Parker-Holder and Shlomi Fruchter explained in a blog post today. “Despite the challenge, Genie 3 environments remain largely consistent for several minutes, with visual memory extending as far back as one minute ago.”

DeepMind believes that Genie 3 could lend itself to training embodied agents. Those are AI agents designed to power autonomous systems such as industrial robots. Often, such algorithms are trained in simulations of the real-world environments they will be expected to navigate.

DeepMind put Genie 3’s embodied agent training features to the test using an AI model called SIMA. The latter algorithm is designed to autonomously perform tasks in virtual environments. During the test, DeepMind researchers successfully instructed SIMA to perform a series of actions in environments generated by Genie 3.

“In each world we instructed the agent to pursue a set of distinct goals, which it aims to achieve by sending navigation actions to Genie 3,” Parker-Holder and Fruchter wrote. “Like any other environment, Genie 3 is not aware of the agent’s goal, instead it simulates the future based on the agent’s actions.”

Initially, DeepMind will make Genie 3 accessible to a limited number of testers as a research preview. The Alphabet unit is currently studying how it could make the AI available to more users in the future.

Image: DeepMind