This paper addresses the slow, strictly sequential token generation inherent to autoregressive language models by introducing diffusion-augmented LLMs that define an autoregressive model distribution while using diffusion to draw multiple tokens in parallel. The approach decouples model parameters into autoregressive weights and lightweight diffusion weights, enabling a family of samplers to achieve lossless acceleration without a separate draft model or any loss of output quality. The resulting models demonstrate up to 3x speedups over autoregressive baselines while outperforming competing approaches such as speculative decoding and standalone diffusion LLMs on coding, tool-use, and reasoning benchmarks.
