Google Research explains how diffusion models generate novel images instead of memorizing training data. The research shows that this “creativity” arises mathematically because neural networks learn smoothed versions of score functions, causing generated data points to interpolate between training samples along hidden data manifolds. The team demonstrates that this smoothing effect comes from both explicit regularization techniques, such as weight decay, and implicit regularization inherent to gradient-based training.
