DDPM on CIFAR-10: a paper-faithful reproduction (FID 3.14)
A from-scratch, paper-faithful reproduction of DDPM (Ho et al., 2020) for unconditional 32×32 CIFAR-10, trained on a single RTX 5090. It reaches a legacy-TF FID of 3.139 (50k samples, 1000-step ancestral sampling, best of 3 seeds), matching the paper's 3.17.
These are the EMA weights (decay 0.99995) of the 700k-step snapshot, the best snapshot of the run. The model is the baseline entry of a single-GPU diffusion speedrun benchmark: https://github.com/Euswbnix/DDPM-Diffusion-Model.
64 uncurated samples: DDPM-1000 ancestral sampling, seed 0.
Results
FID is computed with clean-fid in legacy_tensorflow mode against CIFAR-10 train
statistics, on 50k samples from DDPM-1000 ancestral sampling with EMA 0.99995. Each
record uses exactly three distinct sampling seeds and reports the minimum.
clean-mode FID is given as a disclosure.
| Snapshot | Training time | FID (min of 3) | FID per seed (0 / 1 / 2) | clean-FID (min) |
|---|---|---|---|---|
| 600k | 8.24 h | 3.1912 | 3.1912 / 3.2637 / 3.2757 | 3.8690 |
| 700k (this model) | 9.61 h | 3.1385 | 3.1385 / 3.2275 / 3.2149 | 3.8051 |
| 800k (paper budget) | 10.98 h | 3.1500 | 3.1500 / 3.1824 / 3.1643 | 3.8349 |
Training time is on-clock wall time from process start to the snapshot save. It
includes torch.compile warmup, data loading and EMA updates, and excludes
evaluation.
- The 700k snapshot beats the paper's full 800k-step budget on both FID and time: this recipe plateaus near FID 3.14 by 700k steps.
- The time to reach FID 3.17 falls between 8.24 h and 9.61 h. The 600k snapshot misses the target on all three seeds; the 700k snapshot clears it.
- FID gaps below about 0.04 are within seed-to-seed noise.
The raw evaluation files are in eval/.
Usage
You need torch, safetensors and huggingface_hub. The architecture and the
samplers are the two Python files in this repository.
import json, sys
import torch
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
repo = "euswbnix/ddpm-cifar10"
for f in ("unet.py", "diffusion.py"):
hf_hub_download(repo, f, local_dir=".")
sys.path.insert(0, ".")
from unet import UNet
from diffusion import GaussianDiffusion
cfg = json.load(open(hf_hub_download(repo, "config.json")))
model = UNet(
ch=cfg["ch"], ch_mult=tuple(cfg["ch_mult"]), num_res_blocks=cfg["num_res_blocks"],
attn_res=tuple(cfg["attn_res"]), dropout=cfg["dropout"], img_res=cfg["img_res"],
)
model.load_state_dict(load_file(hf_hub_download(repo, "model.safetensors")))
device = "cuda" if torch.cuda.is_available() else "cpu"
model.to(device).eval()
diffusion = GaussianDiffusion(cfg["T"], cfg["beta_start"], cfg["beta_end"]).to(device)
gen = torch.Generator(device).manual_seed(0)
x = diffusion.p_sample_loop(model, (16, 3, 32, 32), device, generator=gen) # in [-1, 1]
images = (x.clamp(-1, 1) + 1) / 2 # in [0, 1]
Use the ancestral sampler (p_sample_loop) to match the reported numbers. Few-step
DDIM sampling hurts this ε-prediction, linear-β model badly: in 10k-sample checks,
DDIM with 100 steps scored about 8 FID where DDPM-1000 scored about 5.
Training details
- Architecture: 35.7M-parameter U-Net as in the official implementation. Base width 128, channel multipliers (1, 2, 2, 2), two residual blocks per resolution, self-attention at 16×16, dropout 0.1, sinusoidal timestep embedding. Output layers are zero-initialized; all other weights use Xavier-uniform initialization.
- Diffusion: T = 1000, linear β from 1e-4 to 0.02, ε-prediction, simple MSE loss.
- Optimization: Adam, learning rate 2e-4 with a 5k-step linear warmup, gradient clipping at 1.0, batch size 128, random horizontal flips, seed 0.
- EMA: four decay rates tracked during training. 0.99995 gave the best FID and is the one released.
- Systems: bf16 autocast,
torch.compile, PyTorch 2.11, one RTX 5090. - Sampling: DDPM ancestral, 1000 steps, variance σ² = β, predicted x₀ clipped to [-1, 1].
Limitations
- Unconditional CIFAR-10 at 32×32 only. This is a research and benchmark artifact.
- Sampling takes 1000 network evaluations per image.
Citation
@inproceedings{ho2020denoising,
title = {Denoising Diffusion Probabilistic Models},
author = {Ho, Jonathan and Jain, Ajay and Abbeel, Pieter},
booktitle = {Advances in Neural Information Processing Systems},
volume = {33},
pages = {6840--6851},
year = {2020}
}