Google Unveils Genie 4 World Model with Synced Audio and Real-Time Interaction
Google DeepMind has released Genie 4, a research preview world model capable of generating high-fidelity virtual environments with synchronized audio and real-time interactive editing. By integrating deeply with Gemini, the model doubles scene coherence to two minutes and demonstrates potential in robotic simulation testing. Despite limitations like detail drift over time, its capabilities for interactive learning and film prototyping are drawing significant industry attention.
Full Report
IT Home reported on October 28 that Google DeepMind launched its latest world model, Genie 4, on Tuesday. Currently in research preview, this new model generates the most realistic and interactive virtual worlds to date, allowing users to modify scenes by inputting prompts during simulation. The model also generates real-time audio synchronized with the virtual world, including music, ambient sounds, and interactive dialogue.
IT Home noted that just months before the release of Genie 4, AI video startup Runway launched its second world model, GWM Worlds 2. This is an interactively runnable model that responds dynamically to user inputs and supports audio generation. Meanwhile, consolidation trends are emerging in the world model sector. World Labs, founded by Fei-Fei Li, was acquired by AMD; Amazon is in talks to acquire Decart; and Rivian recently acquired Third Dimension to build internal simulation capabilities for autonomous driving testing.
The launch of Genie 4 is Google's latest move to maintain its lead in this highly competitive field.
Over a year ago, Google released its predecessor, Genie 3, as a research preview. Beyond significantly improved visual realism and new audio capabilities, Genie 4 supports free-form editing operations, such as adjusting weather and camera angles in real time, or adding characters and objects. It can maintain coherent, unchanging scenes for up to two minutes—twice the duration of the previous model—with better physical effects and more lifelike virtual characters.
Integration between Google's large language model Gemini and Genie 4 has also deepened. Gemini acts as the director, responsible for planning steps and validating frames, while Genie 4 handles the generation of visual content. In a research experiment related to the Gemini robotics project, the Gemini 4 Argon model decomposed tasks into multiple steps, which were then simulated in real time by Genie 4. This experiment demonstrated a viable approach: validating relevant scenarios in a simulation environment before conducting robot tests in the real world.
Tejas Kulkarni, Director of Research at Google, stated in a briefing: "We will increasingly see hybrid world models like these in the future, where Gemini writes code and text, and Genie 4 reads this content in real time to simulate reality at different levels of abstraction."
Like other world models, its primary commercial use cases include gaming, interactive learning, and robotics. A conceptual example of interactive learning involves users entering a painting depicting a historical scene and interacting with the figures within it about the events portrayed.
However, "trusted testers"—the first group of users able to experience Google's cutting-edge products—have discovered a potential new use case: film prototyping.
Diego Rivas, an engineer at Google DeepMind, said in a media briefing: "The value of real-time capabilities is particularly evident in scenarios requiring sustained creative flow. For instance, it applies to ideation phases, storyboarding, and any stage where rapid creative bursts are needed."
Google has also introduced Veo, an AI video generation model capable of producing high-definition short clips from text, image, or video prompts, complete with synchronized audio. This tool has seen rising popularity among filmmakers.
Rivas added that users may eventually leverage Gemini Omni, a multimodal large model, to transform prototype assets into formal production materials. Gemini's full modality capabilities allow for video generation based on images, audio, and text. However, he noted that these capabilities are still some distance from widespread deployment.
Throughout the briefing, Rivas repeatedly emphasized that Genie 4 remains in its early stages, and the final experience of the complete product is yet to be determined. The team acknowledged existing limitations in Genie 4: as simulation runtime increases, details and objects tend to drift. They have not yet found a robust method to ensure accurate audio synchronization while keeping text within the virtual world clearly legible.
Currently, Genie 4 is available for research preview to trusted testers. Over the coming weeks, access will gradually expand to Google AI Ultra subscribers and participants in the Genie Project—an AI tool launched by Google that generates interactive virtual worlds from text or image prompts. Early partners using the Gemini API and Google AI Studio will also gain access.