Google released DiffusionGemma, a 26B mixture-of-experts model that generates entire blocks of text simultaneously rather than processing tokens sequentially like traditional language models. The model uses an iterative refinement approach similar to image diffusion, starting with placeholder tokens and progressively locking in correct ones across multiple passes. By generating 256 tokens in parallel with bidirectional attention on dedicated GPUs, DiffusionGemma achieves up to 4x faster inference while offering advantages for non-linear tasks such as code infilling and markdown formatting.
