Google’s DiffusionGemma open AI model gets 4x speed boost

Google DeepMind has unveiled a new addition to its Gemma 4 open model family, and it breaks the mold in a significant way. DiffusionGemma does not generate text one token at a time like most AI models. Instead, it produces entire blocks of text in parallel, a shift that Google claims makes it faster and more efficient when running on local hardware, from an Nvidia DGX system to a standard gaming GPU.

The majority of AI models are autoregressive, meaning they generate text sequentially from left to right. DiffusionGemma, however, operates more like an image generation model. It starts with a field of placeholder tokens and iteratively “denoises” them across multiple passes, refining likely tokens to improve estimations of others. At the end of this process, the model finalizes its output in one large, coherent block , essentially a denoised text canvas.

In terms of scale, DiffusionGemma is substantial for an open model. It is a Mixture of Experts (MoE) architecture with 26 billion total parameters, but only 3.8 billion are activated during inference. This design allows it to fit comfortably within the 18GB memory of a high-end consumer GPU. In benchmark testing with an RTX 5090, DiffusionGemma achieved around 700 tokens per second. On a single Nvidia H100 AI accelerator, that rate jumps to over 1,000 tokens per second , roughly four times the output of similarly sized autoregressive Gemma models.

This parallel approach shifts the computational bottleneck from memory bandwidth to raw compute, enabling the generation of up to 256 tokens simultaneously. Google highlights measurable gains in non-linear tasks such as in-line editing, molecular sequencing, and mathematical graphing. The model’s ability to continuously self-correct large sets of tokens makes it particularly effective for challenges like solving Sudoku puzzles, a task that stumps standard autoregressive models because each token depends on future context.