Google DeepMind ships DiffusionGemma, an open-weights 26B MoE that denoises 256 tokens at a time
DiffusionGemma 26B-A4B is the first production-grade open text-diffusion model from a frontier lab. Built on the Gemma 4 backbone with 8 of 128 experts active per token and 3.8 billion active parameters, it abandons sequential autoregression and instead refines a 256-token canvas in parallel through iterative denoising. Throughput lands at more than 1,000 tokens per second on a single NVIDIA H100, 700-plus on an RTX 5090, and up to 2,000 tokens per second on a DGX Station — roughly 4x an autoregressive model at the same parameter budget. The model is multimodal across text, image and video inputs with a 256K-token context window and 140-plus languages, and ships under Apache 2.0 on Hugging Face with day-zero support in Transformers, vLLM, MLX and Unsloth. Google is direct in the model card that quality lags Gemma 4 in absolute terms and recommends Gemma 4 for production today; the release is positioned as an experimental open building block for in-line editing, code infill and real-time agents where denoising's self-correction loop matters more than absolute log-likelihood.
Google DeepMind · 2026-06-10 · Source ↗ ·
Google DeepMind ships DiffusionGemma, an open-weights 26B MoE that denoises 256 tokens at a time
DiffusionGemma 26B-A4B is the first production-grade open text-diffusion model from a frontier lab. Built on the Gemma 4 backbone with 8 of 128 experts active per token and 3.8 billion active parameters, it abandons sequential autoregression and instead refines a 256-token canvas in parallel through iterative denoising. Throughput lands at more than 1,000 tokens per second on a single NVIDIA H100, 700-plus on an RTX 5090, and up to 2,000 tokens per second on a DGX Station — roughly 4x an autoregressive model at the same parameter budget. The model is multimodal across text, image and video inputs with a 256K-token context window and 140-plus languages, and ships under Apache 2.0 on Hugging Face with day-zero support in Transformers, vLLM, MLX and Unsloth. Google is direct in the model card that quality lags Gemma 4 in absolute terms and recommends Gemma 4 for production today; the release is positioned as an experimental open building block for in-line editing, code infill and real-time agents where denoising's self-correction loop matters more than absolute log-likelihood.
Google DeepMind · 2026-06-10 · Source ↗ ·
Google DeepMind on June 10 released DiffusionGemma 26B-A4B, an open-weights text diffusion mixture-of-experts model that the team frames as a public building block rather than a competitor to Gemma 4 on absolute capability. The architecture takes the Gemma 4 backbone — 30 layers, 128 experts, 8 active per token, plus one shared expert — and grafts on a discrete text diffusion head. Generation no longer runs left-to-right one token at a time. Instead, the decoder applies bidirectional attention over a 256-token canvas and iteratively denoises the canvas through multiple forward passes, finalizing roughly 15 to 20 tokens per pass. An autoregressive encoder processes and caches the prompt context so the generation work is concentrated in the parallel decoder.
The throughput numbers are the marketing line. A single NVIDIA H100 sustains more than 1,000 tokens per second; a consumer-grade RTX 5090 holds 700-plus; an NVIDIA DGX Station pushes the model to 2,000 tokens per second. NVIDIA's day-zero blog says that is "roughly 4x faster than an equivalent autoregressive model" at the same parameter budget. The total parameter count is 26 billion, but only 3.8 billion are active per token thanks to the sparse MoE routing, and the quantized weights fit inside the 24 GB VRAM ceiling of an RTX 4090 or 5090. Context window is 256K tokens. The vision encoder adds about 550 million parameters and accepts interleaved text, image and video inputs up to 60 seconds. The model speaks 140-plus languages.
How parallel denoising changes what the model can do
The architectural shift matters for a category of tasks autoregressive models handle poorly. Because the decoder applies bidirectional attention across the entire canvas, the model can revise tokens it has already committed to, by re-noising and re-denoising regions that look inconsistent with later context. Autoregressive transformers cannot do this — once a token is sampled, it is fixed. Google's developer guide leans into use cases where that property pays off: in-line editing of existing text, code infill where the model has to thread new content between fixed anchors above and below, and real-time interactive applications where the model can keep refining a draft as the user types.
"DiffusionGemma abandons the sequential, token-by-token process of typical autoregressive Large Language Models."— Google DeepMind, DiffusionGemma model page
The model card is unusually honest about the trade. On MMLU Pro the score lands at 77.6%, GPQA Diamond at 73.2%, LiveCodeBench v6 at 69.1%, and MMMU Pro at 54.3% for vision tasks. Those numbers sit below standard Gemma 4 at the same parameter budget. Google states it directly: "output quality is lower than standard Gemma 4," with a recommendation that production workloads still use Gemma 4 today. The trade is that the bottleneck moves from memory bandwidth to compute, which is exactly the regime modern accelerators are designed for, so the same hardware delivers more usable tokens per second.
What ships on day one
The weights are on Hugging Face at `google/diffusiongemma-26B-A4B-it` under Apache 2.0. Day-zero support lands in Hugging Face Transformers, vLLM, Apple's MLX runtime and Unsloth, with llama.cpp work in progress. The Google Cloud Model Garden carries the model behind the standard Vertex API, and NVIDIA NIM offers a containerized inference endpoint optimized for Blackwell's NVFP4 quantization. Unsloth simultaneously published a GGUF quantization at `unsloth/diffusiongemma-26B-A4B-it-GGUF` for local self-hosting.
The licensing posture matters. Apache 2.0 is the maximally permissive open-weights license Google has used; previous Gemma releases shipped under the Gemma terms. Pairing the model with Apache 2.0 invites third-party fine-tunes, distillations and downstream commercial products without a separate license agreement. Combined with the lower active parameter count, that lowers the cost floor of running an in-house diffusion text model from "you need an A100" to "you can use the RTX card you already have."
The training cutoff and safety posture
The pre-training mixture spans web documents in 140-plus languages, code, math and images with a January 2025 cutoff. Google ran the standard preprocessing pipeline — CSAM filtering, sensitive personal data removal, quality and safety filtering. The card spells out the limitations: the model depends on training data quality, struggles with ambiguity and sarcasm in nuanced language, can generate factually incorrect text, and has limited common-sense reasoning. There is built-in reasoning and function-calling support, with variable image-token budgets between 70 and 1,120 tokens per image to trade resolution against latency.
The strategic read: this is the first time a frontier lab has published an open, production-runnable diffusion language model with the same release rigor as a standard autoregressive Gemma. The 4x throughput claim is real on paper and will get stress-tested across the next month on agent loops, code-completion benchmarks and edit-in-place applications where the bidirectional canvas actually changes what the system can do. The quality gap to Gemma 4 is real too, and the model card's honesty about that gap is meant to keep production users from defaulting to DiffusionGemma when they should still be on Gemma 4.