Paper
Denoising Diffusion Probabilistic Models
Jonathan Ho, Ajay Jain, Pieter Abbeel
arXiv:2006.11239 · 0▲ · cs.LG, stat.ML
View on arXiv →Premium readers get an interactive explainer for this paper — a figure you can poke at, not just read.
What it is
This paper introduces Denoising Diffusion Probabilistic Models (DDPMs), a generative model that learns to reverse a fixed Markov chain that gradually adds Gaussian noise to images. The key practical simplification is training a U-Net to predict the noise added at each timestep using a plain weighted mean-squared-error objective, which the authors show is equivalent to denoising score matching over multiple noise levels.
Why it matters
This is the paper that made diffusion models competitive with GANs on image quality, establishing the training recipe (noise prediction, fixed variance schedule, U-Net backbone) that later systems like Stable Diffusion and DALL-E 2 built on. The tradeoff to note: sampling requires 1000 sequential network evaluations, so generation is slow compared to GANs, and log-likelihoods are not competitive with other likelihood-based models.
Practical takeaway
If you want to understand why modern image generators are built the way they are, this is the source of the epsilon-prediction training objective and the multi-step denoising sampler. Expect sampling latency to be the main pain point (T=1000 steps per image), which later work addressed with faster samplers.
Key result
On unconditional CIFAR10, the model achieved an FID of 3.17 and Inception score of 9.46, beating prior unconditional models and most class-conditional ones at the time. On 256x256 LSUN, sample quality was comparable to ProgressiveGAN (e.g. LSUN Bedroom FID 4.90 vs ProgressiveGAN 8.34, though StyleGAN still scored better at 2.65).
Subscribe
Get the next issue.
Free. One email a week. Unsubscribe any time: no account, no dark patterns.