Item: DiffusionGemma 26B
Author: Elena Marchetti

Every language model released in the past several years generates text the same way: one token at a time, left to right, each word committed before the next begins. DiffusionGemma doesn't work like that. Google DeepMind's 26B open-weight model released June 10 starts each output block as random noise and refines an entire 256-token canvas in parallel, borrowing iterative denoising from image generation and applying it to language at production scale. Measured independently by the vLLM team, it hits 1,008 tokens per second on a single H100. The cost is a measurable quality regression on every major benchmark except document parsing. Whether that trade makes sense depends completely on what you're building.

TL;DR

7.5/10 - the fastest open-weight LLM available, with real benchmark costs
Bidirectional attention during generation enables code infilling and constrained output that autoregressive models simply can't do at this speed
5-19 point gaps vs Gemma 4 26B across reasoning, math, and vision benchmarks
Best fit for real-time local apps and code editors; skip it if answer quality is your binding constraint

How It Works

The architectural break from autoregressive generation is worth understanding before weighing the tradeoffs.

Standard language models predict one token at a time, each conditioned on everything written before it. The left-to-right constraint means a model producing token 47 has no knowledge of what tokens 48-256 will be. For most text tasks, that's fine. For tasks where the end of a sentence constrains the beginning - crossword fills, code infilling, JSON with required fields - it creates real problems.

DiffusionGemma uses what Google calls Uniform State Diffusion. At generation time, the model maintains a 256-token canvas initialized to random noise. Through multiple denoising passes, confident tokens solidify first, and those committed values help resolve adjacent positions. Each forward pass uses bidirectional attention - every position on the canvas attends to every other position simultaneously. An autoregressive model generating token 12 can't see tokens 13 through 256; DiffusionGemma's denoiser sees all of them at once.

For sequences longer than 256 tokens, the model processes blocks sequentially. Once a block fully converges, it gets committed to the KV cache and the next block starts from noise again. This block-autoregressive structure preserves the full 256K context window inherited from the Gemma 4 26B backbone without losing the bidirectional advantage within each block.

Entropy-bound early stopping reduces average latency further. On simple queries where tokens converge fast, the model stops after fewer denoising steps rather than running the full 48-step budget. Complex outputs get more passes. In practice, average throughput on mixed workloads runs higher than the peak numbers suggest.

Speed Numbers

The throughput claims here are more credible than most because the vLLM team ran their own benchmarks before announcing first-class support.

DiffusionGemma running locally on NVIDIA hardware, showing the parallel diffusion inference pipeline in action DiffusionGemma on NVIDIA RTX hardware via the vLLM serving stack, which became the first production inference framework to natively support a diffusion LLM. Source: blogs.nvidia.com

At batch size 1 on a single H100 with FP8 quantization, vLLM independently measures 1,008 tokens per second. H200 reaches 1,288 tokens per second in the same configuration. NVIDIA's RTX 5090 hits 700+ tokens per second. The DGX Station, running multiple GPUs, reaches up to 2,000 tokens per second.

One number that undercuts the excitement: the DGX Spark, NVIDIA's compact desktop machine with 128GB unified memory, delivers 150 tokens per second. That's still usable for interactive chat, but it's less than a quarter of the H100 result and a fraction of what the RTX 5090 achieves. Developers planning a local DGX Spark setup should adjust expectations.

There's also a batch size constraint that matters for production. DiffusionGemma's speed advantage is strongest at batch sizes 1-8, covering single-user and small-concurrency scenarios. At batch size 32 and above, autoregressive models recover their footing because they can share KV cache across concurrent requests - an efficiency mechanism that DiffusionGemma's bidirectional attention architecture can't copy in the same way. High-concurrency multi-user serving isn't where this model shines.

The Benchmark Gap

Google is clear about the quality tradeoff. DiffusionGemma's output quality is lower than Gemma 4 26B on every benchmark they tested, with document parsing as the sole exception.

Benchmark	DiffusionGemma 26B	Gemma 4 26B	Gap
MMLU Pro	77.6%	82.6%	-5pt
AIME 2026	69.1%	88.3%	-19pt
LiveCodeBench v6	69.1%	77.1%	-8pt
GPQA Diamond	73.2%	82.3%	-9pt
MMMU Pro	54.3%	73.8%	-19pt
MATH-Vision	70.5%	82.4%	-12pt

The AIME 2026 and MMMU Pro gaps are the ones to study. A 19-point drop on competition mathematics isn't noise - it reflects a genuine limitation of parallel block generation on multi-step reasoning tasks where each step depends tightly on the previous one. Sequential causal dependency is exactly where autoregressive models hold up, and exactly where the diffusion approach pays a real cost.

The GPQA Diamond result tells a different story. At 73.2%, DiffusionGemma remains competitive with many frontier models from 2025. The benchmark gap exists, but the absolute number isn't weak for a model running four times faster. Workloads led by information retrieval, summarization, and structured output will find the quality floor more tolerable than the table makes it appear at first glance.

Document parsing, where DiffusionGemma reportedly beats Gemma 4, is the clearest sign that bidirectional attention carries real advantages in the right domain. OCR, table extraction, and structured document analysis all benefit from seeing the full context simultaneously.

The full spec comparison is in the DiffusionGemma 26B model card, alongside detailed notes on what each benchmark actually tests.

What Bidirectional Attention Unlocks

The benchmarks show where DiffusionGemma loses. Bidirectional attention is where it wins in ways a standard autoregressive model can't match.

DiffusionGemma 26B on Hugging Face, available as google/diffusiongemma-26B-A4B-it with day-one support in vLLM, HF Transformers, SGLang, and MLX. Source: huggingface.co

Code infilling is the most practical example. An autoregressive model producing a function body has no view of how that body ends. A model writing the middle of a code block with bidirectional attention sees the full 256-token canvas from the first denoising pass, which matters when an early variable declaration needs to match a later usage or when an opening bracket needs to pair with something many lines below.

Google's Sudoku demonstration is the sharpest illustration. After supervised fine-tuning on a synthetic Sudoku dataset using a simple JAX training recipe, the fine-tuned DiffusionGemma variant solved 80% of puzzles correctly. A standard autoregressive model starting from the same checkpoint scored 0%. The reason is architectural: each digit in a Sudoku must satisfy row, column, and box constraints at once. A model that can only look backward during generation has no way to enforce those constraints on tokens it has already committed. DiffusionGemma can, because it re-noises and re-refines uncertain positions rather than locking them in.

The same advantage applies to JSON schemas with required fields, markdown tables needing consistent column alignment, and any output format where the beginning must agree with the end. For IDE integrations, documentation generators, and data extraction pipelines, that's a truly useful property.

Deployment

Hardware requirements are straightforward for a 26B model: 18GB VRAM at minimum with FP8 quantization, which puts it in RTX 4090 or RTX 5090 territory for local single-user inference. H100 or H200 for multi-user production serving.

Framework support at launch is solid but incomplete. vLLM integration works in production, with FP8 and NVFP4 quantized checkpoints available from the RedHatAI hub. Hugging Face Transformers supports it via the DiffusionGemmaForBlockDiffusion class. SGLang and MLX both work. Fine-tuning options include Unsloth LoRA adapters, NVIDIA NeMo, and Google's Hackable Diffusion JAX toolbox with provided SFT recipes.

llama.cpp support is an unmerged pull request as of the June 10 release date. Ollama doesn't support it. If your local inference workflow depends on either of those - as many developers' workflows do - DiffusionGemma isn't plug-and-play yet.

One concrete deployment issue that caught teams off guard: the initial NVIDIA NIM endpoint shipped with a context window of 8,192 tokens instead of the model's 256K capacity. That's a configuration error rather than a model limitation, but frameworks requiring 64K+ context discovered the discrepancy immediately. Verify the context limit on whatever managed endpoint you use before committing to it.

Competitor context: Inception's Mercury 2 reaches similar throughput numbers as a diffusion-based model, but it's API-only. DiffusionGemma offers the full open-weight stack at no cost, which is a meaningful distinction for teams that need data residency or want to fine-tune.

Strengths

1,000+ tokens/sec on H100 - measured by the vLLM team independently, not just Google's own benchmarks
Bidirectional attention enables code infilling and constrained generation that no autoregressive model can do at equivalent speed
Apache 2.0 license with no usage restrictions; full self-host with commercial deployment allowed
Fits on a RTX 4090 with FP8 quantization - the fastest open-weight option in that VRAM range
Adaptive early stopping reduces average latency on simple queries without any code changes needed
Native vLLM, HF Transformers, SGLang, and MLX support at launch
Full multimodal input (text, image, video up to 60 seconds) inherited from Gemma 4

Weaknesses

5-19 point benchmark gaps vs Gemma 4 26B on every major evaluation
AIME 2026 at 69.1% vs 88.3% - the multi-step math gap is large and reflects a genuine architectural constraint
Text output only; doesn't generate images or video
llama.cpp and Ollama support absent at launch; ecosystem maturity lags behind autoregressive alternatives
Speed advantage collapses at batch size 32+ where KV cache reuse benefits autoregressive models
Initial NIM deployment shipped with 8,192 token context rather than the model's actual 256K

Verdict

DiffusionGemma earns a 7.5/10.

For throughput-constrained local inference at batch size 1-8, it's the best open-weight option available. The bidirectional attention advantage on code infilling and constrained generation is real and not something you get by simply choosing a faster autoregressive model. Apache 2.0 with no usage restrictions makes it truly deployable.

The benchmark gaps are equally real. A 19-point drop on complex math and 12 points on MATH-Vision don't disappear because the model is fast. For assistant applications, general Q&A, and anything requiring sustained multi-step reasoning, Gemma 4 26B remains the cleaner choice. The tooling gap - no Ollama, no llama.cpp - means friction for many developers is higher than the headline throughput numbers suggest. That changes as the ecosystem matures, but in June 2026, vLLM or Hugging Face Transformers is the only reliable path to production.

Pick DiffusionGemma when you need speed and can tolerate the quality floor. Pick Gemma 4 when quality drives the decision.

DiffusionGemma 26B Review: 4x Faster, Real Tradeoffs