Breaking

Google Rolls Out Gemini Omni That Uses In-Video Audio to Steer Output

July 7, 2026 at 18:51 EDT

Media Generation
Foundation Models

Google has introduced a capability in its video-generation AI "Gemini Omni" that uses audio contained in an input video to steer the final output video. Integrated into the creative tool "Google Flow," it enables video-making guided not only by text and images but also by audio.

Google I/O 2026 · Google DeepMind

Gemini Omni: Edit Video by Talking to It

Google's new native multimodal model steers video generation using the audio inside your clip — say what you want changed, and it edits conversationally. Positioned to replace the Veo series as an "any-to-any" model.

4→1

Text, image, audio & video in — one high-quality video out

~10s

Typical output length, with native audio generation

Surfaces: Gemini app, Flow, YouTube Shorts & API

The Conversational Edit Loop

Speak the instruction
Request spoken in the video clip

→

Model analyzes audio
Uses it as an editing guide

→

Refine over turns
Consistency kept across edits

Praised for

Physical consistency — viscosity, bubbles
Intuitive editing by talking to a video
Style transfer, VFX & character insertion
Fast short-form output for Shorts & TikTok

Watch out for

Consistency drops in longer videos
Results depend heavily on prompts
Iterative fine-tuning often needed
API cost & rate limits to manage

Built-in Transparency

Every output carries a SynthID watermark and C2PA metadata to flag AI-generated content — framed as a video counterpart to the conversational image editor Nano Banana.

Continue reading

The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.