Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseAPI / Product · ActionableTG: @data_secrets (Data Secrets)→ original

Google Releases Genie 4: A Neural Network That Builds Worlds With Sound and Live Dialogue in Real Time

Google has released Genie 4, a neural network that generates an interactive world in real time, where you can act as you would in a game. Along with each frame, it produces music, ambient noise and character dialogue that adapt to the player's actions. The scene can be changed with prompts on the fly, for example by turning on rain or adding characters. For developers of robots and agents, it serves as a virtual testing ground where checks don't have to be set up by hand.

AI-processed from TG: @data_secrets (Data Secrets); edited by Hamidun News
Google Releases Genie 4: A Neural Network That Builds Worlds With Sound and Live Dialogue in Real Time
Source: TG: @data_secrets (Data Secrets). Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceTG: @data_secrets (Data Secrets) ↗
Editor-in-Chief
Key metric #1Gemini Omni handles the realistic look and behavior of the world
Gemini 3.8 Flash acts as the agentit plans steps and revises the plan
Key metric #3In the demo, the humanoid Apollo from Apptronik plays checkers inside the world

Google has released Genie 4, the new generation of its "world model." That is the term for a neural network that creates an interactive environment in real time: you can move around in it, change things and get reactions, as in a game. In Genie 4 the world comes with sound from the start: music, ambient noise and even character dialogue are generated together with the picture, and the dialogue adapts to what the user does. The previous version came out more than a year ago, in August 2025, and with the new one Google has focused on realism, multimodality, improved physics and interactivity.

Why does this matter in practice? Imagine you need to check how a robot or AI agent will behave in a hundred situations: on a street in the rain, in a room with unfamiliar people, in a space where a new object has suddenly appeared. Building such scenes by hand takes a long time and costs a lot. Genie 4 offers a different route: describe the situation in words and get an environment in which the system has to act.

A World With Sound and Live Dialogue

The first thing that stands out is the sound. Earlier world generators mostly produced pictures, and audio had to be added separately. Here, music, ambient noise and dialogue are created at the same time as the video. The lines are not recorded in advance: if you approach a character or do something in the scene, the response takes that into account.

The second is on-the-fly direction. The scene can be changed with prompts while it is running: change the weather, add characters or objects. In effect, it feels like a game engine in which, instead of a level editor, you write down what you want to see.

How It Works Inside

Under the hood, several Gemini models work together, each with its own role:

  • Gemini Omni is responsible for the realistic appearance and behavior of the world;
  • other models keep the simulation coherent so the world does not fall apart as things unfold;
  • Gemini 3.8 Flash works as an agent in the loop, a kind of brain for the system.

The last role is easiest to understand through an example. Say you set the goal "throw a party in the yard." The agent breaks it into steps, checks whether they are done, and revises the plan if, midway through the scene, you decide the evening should be rainy and more guests have arrived. The world is thus driven by a plan that can be changed at any moment, while the frames remain consistent with one another.

What It Offers Developers and Product Managers

According to Google's plan, such environments can be used to test robots and agents. For the demonstration, the company showed the humanoid Apollo from Apptronik playing checkers inside a generated world. Instead of an expensive physical testing ground or manual simulator setup, the scenario is defined in text.

What can be taken from this:

  • Agent testing is gradually shifting toward generative environments: scenarios are created and changed with text, without separately developing a simulator.
  • Sound and dialogue inside the simulation open the way to testing voice and multimodal assistants under more lifelike conditions.
  • The scheme of "a generator model, controller models and a planner agent" is a useful reference for your own systems: separating roles helps keep a long session coherent.

What Remains to Be Verified

For now this is an announcement and a demonstration. Checkers is a telling but simple scenario with clear rules. For serious robot testing, what matters is how closely the physics of the generated world matches reality, how long the world stays coherent, how much latency it has in responding to actions, and whether the same scenario can be repeated so that versions of an agent can be compared with each other. The answers to these questions will determine whether Genie 4 becomes a working testing tool or remains an impressive showcase.

The direction is nonetheless clear: Google is betting on simulation as the platform where future agents and robots are trained and tested. Those building such systems should watch for Genie 4's availability and compare it with the familiar simulators. Details are published on the Google DeepMind blog.

Story

Key Questions Answered

What are the key details on A World With Sound and Live Dialogue?

The first thing that stands out is the sound. Earlier world generators mostly produced pictures, and audio had to be added separately. Here, music, ambient noise and dialogue are created at the same time as the video. The lines are not recorded in advance: if you approach a character or do something in the scene, the response takes that into account. The second is on-the-fly direction. The scene can be changed with prompts while it is running: change the weather, add characters or objects.

How It Works Inside?

Under the hood, several Gemini models work together, each with its own role: Gemini Omni is responsible for the realistic appearance and behavior of the world; other models keep the simulation coherent so the world does not fall apart as things unfold; Gemini 3.8 Flash works as an agent in the loop, a kind of brain for the system. The last role is easiest to understand through an example.

What It Offers Developers and Product Managers?

According to Google's plan, such environments can be used to test robots and agents. For the demonstration, the company showed the humanoid Apollo from Apptronik playing checkers inside a generated world. Instead of an expensive physical testing ground or manual simulator setup, the scenario is defined in text. What can be taken from this: Agent testing is gradually shifting toward generative environments: scenarios are created and changed with text, without separately developing a simulator.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…