Google Unveils Genie 4 World Model with Real-Time NL Interaction and Native Audio
Google DeepMind released Genie 4, a major research milestone transitioning from static video generation to dynamic world models with real-time controllability, native audio, and physics simulation. Users can alter weather, camera angles, or characters via natural language instructions, triggering context-aware evolution rather than regeneration. Despite early-stage limitations like detail drift, the model demonstrates 'AI director' potential and plans to open access via Gemini API.
Full Report
IT Home reported on October 28 that the Google DeepMind team released a blog post yesterday (October 27) announcing the launch of Genie 4, a world model positioned as a "major research milestone." The model achieves core breakthroughs in native audio generation, real-time natural language modification, and more realistic physics and character dynamics, upgrading world models from one-off image generation to dynamic simulated worlds that are controllable in real time, audible, and capable of reasoning.
For users, the biggest highlight of Google's Genie 4 world model is its ability to "direct" worlds in real time. During simulation runs, users can alter the world using natural language text commands, such as changing the weather, moving the camera, or adding new characters or objects.
Unlike mainstream AI approaches that regenerate an entire video segment, Genie 4 executes dynamic changes by integrating with the current scene context, functioning more like an "interactive virtual sandbox" or an "AI director system." Users can set up an event, observe how the world evolves, and intervene to make adjustments at any time.
In terms of other highlights, Genie 4 is DeepMind's first model to natively generate audio within real-time world simulations, with audio and visuals produced synchronously in the same process. Additionally, Google states that Genie 4 demonstrates stronger performance in physical scenarios involving aerodynamics and collisions. Character movements are also more natural, allowing for organic reactions to the environment and supporting real-time control. IT Home has attached relevant demonstration videos below:
Google CEO Sundar Pichai posted on his personal account today, stating that Genie 4 is Google's most realistic and interactive world model to date.
In an interview with tech media outlet TechCrunch, Google Research Director Tejas Kulkarni stated that in the future, Gemini will handle text, code, and task planning, while Genie 4 will convert these into interactive worlds in real time, forming a "hybrid world model."
DeepMind engineer Diego Rivas acknowledged that Genie 4 is still in its early stages; long-duration simulations may experience detail drift, and issues regarding audio accuracy and the readability of text within scenes have not yet been fully resolved.
Currently, Genie 4 is available only as a research preview to "trusted testers." Over the coming weeks, access will expand to Project Genie for Google AI Ultra subscribers, as well as early partners of the Gemini API and Google AI Studio.