Breaking

Google Publishes Gemma 4 Technical Report Detailing Encoder-Free Design

July 7, 2026 at 17:03 EDT

Foundation Models
Open Source
Research & Papers

Google and Google DeepMind have published the technical report for Gemma 4, their family of open-weight multimodal models. Spanning five sizes from E2B to 31B, it lays out the design decisions behind the models, along with their encoder-free architecture and quantization and inference-acceleration techniques.

April 2026 · Google DeepMind

Gemma 4: Open Models Go Encoder-Free

A new family of open-weight, natively multimodal models — from a phone-sized E2B to a 31B workstation model — built on Gemini technology. The headline: a 12B that reads text, images and audio without a separate vision encoder.

256K

context window

140+

languages supported

150M+

cumulative Gemma downloads

Apache 2.0

open license, all sizes

The encoder-free 12B

Instead of routing images through a ~550M vision encoder, the 12B projects RGB patches and audio chunks straight into the LLM — via a lightweight ~35M projection.

~550M

Separate vision encoder

→

~35M

Lightweight projection

≈16× fewer parameters for multimodal input — simpler pipelines, lower latency.

Arena AI — blind human eval (text)

1451

Gemma 4
31B

1438

Gemma 4
26B-A4B

1365

Gemma 3
27B

The 31B ranks roughly 3rd among open models — a marked jump from the previous generation.

Efficiency toolkit

37.5%

smaller global KV cache via sharing & reuse

64GB → 19.2GB

31B compressed with QAT quantization

5:1

local-to-global attention ratio (4:1 on E2B)

Plus p-RoPE partial rotation and an MTP drafter for speculative decoding.

Welcomed

12B runs text + image + audio natively on a 16GB laptop
MoE efficiency and lower barrier to local deployment
VQA, OCR, document understanding, on-device speech & diarization

Open questions

Dense 31B runs slower than Qwen 3.5 on the same hardware
Efficiency-vs-speed tradeoff still debated
Quantization or fine-tuning needed to reach peak performance

Continue reading

The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.