Appendix A — Acronyms & Abbreviations#
How to read this list. Every acronym and abbreviation that appears anywhere in the ten lessons is collected here, expanded in full, and explained in one or two sentences of plain language. The right-hand column points you to the lesson(s) where the term does real work, so you can jump back to its first proper use. Terms are grouped by theme rather than alphabetically, because related ideas are easier to learn together; an alphabetical quick-index follows at the end.
Core model families#
| Acronym | Full form | What it means | Lesson(s) |
|---|
| DDPM | Denoising Diffusion Probabilistic Models | The canonical diffusion model: a fixed forward noising chain plus a learned reverse denoiser, trained by noise-prediction regression. The backbone of the whole course. | 1–9 |
| DDIM | Denoising Diffusion Implicit Models | A non-Markovian reformulation of the same trained model that allows fast and even fully deterministic sampling in far fewer steps. | 1, 3, 8, 9, 10 |
| VAE | Variational Autoencoder | A latent-variable generative model with a learned encoder and decoder trained on the ELBO. A diffusion model is a VAE whose encoder is frozen to the noising chain. | 1, 4, 7, 10 |
| GAN | Generative Adversarial Network | A generator/discriminator pair trained as a min–max game; fast to sample but unstable. The main contrast class to diffusion. | 1, 6, 8, 10 |
| VDM | Variational Diffusion Models | A line of work expressing diffusion entirely through the (log-)signal-to-noise ratio, clarifying which design choices actually matter. | 2, 4, 5, 7 |
| LDM | Latent Diffusion Models | Diffusion run in the compressed latent space of an autoencoder rather than on raw pixels — the recipe behind Stable Diffusion. | 10 |
| NCSN | Noise Conditional Score Network | The score-matching predecessor to DDPM that learns the gradient of the log-density at many noise levels. | 9 |
Objectives & training quantities#
| Acronym | Full form | What it means | Lesson(s) |
|---|
| ELBO | Evidence Lower Bound | A tractable lower bound on the data log-likelihood; maximizing it is the principled training objective behind both VAEs and diffusion. | 1, 4, 5, 7, 10 |
| VLB | Variational Lower Bound | Same object as the ELBO, written $\mathcal L_{\text{vlb}}$; the per-step KL decomposition used as the full diffusion loss. | 4, 7, 8 |
| KL | Kullback–Leibler divergence | An asymmetric measure of how far one distribution is from another, written $D_{\mathrm{KL}}(\cdot\,\|\,\cdot)$. Each diffusion training term is a KL between the true and learned reverse step. | 1, 4, 5, 7, 10 |
| MSE | Mean Squared Error | The squared-distance regression loss; the simplified diffusion objective is just an MSE between true and predicted noise. | 1, 5, 7, 8 |
| MLE | Maximum Likelihood Estimation | Fitting parameters by maximizing the likelihood of the data; the gold standard that the ELBO approximates. | 1 |
| NLL | Negative Log-Likelihood | The quantity actually minimized (lower is better); the ELBO is an upper bound on the NLL. | 4 |
| SNR | Signal-to-Noise Ratio | The ratio $\bar\alpha_t/(1-\bar\alpha_t)$ of preserved signal to injected noise at step $t$; its logarithm is the natural axis for comparing schedules and loss weightings. | 2, 3, 5, 7, 9 |
| FID | Fréchet Inception Distance | A standard sample-quality metric comparing the statistics of generated and real images (lower is better). | 5 |
| LPIPS | Learned Perceptual Image Patch Similarity | A perceptual distance between images used to train the autoencoder in latent diffusion. | 10 |
The continuous / score-based view#
| Acronym | Full form | What it means | Lesson(s) |
|---|
| SDE | Stochastic Differential Equation | The continuous-time limit of the noising chain; diffusion is a particular variance-preserving SDE. | 1, 2, 3, 8, 9, 10 |
| ODE | Ordinary Differential Equation | The deterministic “probability-flow” companion to the SDE that shares its marginals — DDIM is its discretization. | 1, 9, 10 |
| VP | Variance-Preserving | The schedule family (used by DDPM) that keeps the marginal variance bounded near $1$ throughout noising. | 2, 9 |
| VP-SDE | Variance-Preserving Stochastic Differential Equation | The continuous-time SDE whose discretization is the DDPM forward chain. | 2, 9 |
Samplers & solvers#
| Acronym | Full form | What it means | Lesson(s) |
|---|
| DPM-Solver | Diffusion Probabilistic Model Solver | A fast, high-order ODE solver tailored to the diffusion ODE, enabling quality samples in ~10–20 steps. | 9 |
| DEIS | Diffusion Exponential Integrator Sampler | Another exponential-integrator fast solver for the diffusion ODE. | 9 |
| RK | Runge–Kutta | The classical family of ODE integrators (including Heun’s method) used as higher-order samplers. | 9 |
| EDM | Elucidating the Design space of diffusion Models | Karras et al.’s framework that unifies preconditioning, schedules, and weighting; source of the widely used $c_{\text{skip}}/c_{\text{out}}$ design. | 5, 6, 7, 9 |
Architecture & conditioning#
| Acronym | Full form | What it means | Lesson(s) |
|---|
| U-Net | U-shaped (encoder–decoder) Network | The image-to-image convolutional backbone with skip connections that implements the noise predictor $\epsilon_\theta$. | 1, 5–10 |
| DiT | Diffusion Transformer | A transformer-based alternative to the U-Net backbone that scales well at high resolution. | 6, 9, 10 |
| MLP | Multi-Layer Perceptron | A small fully connected network; used here to map the time embedding into per-block conditioning signals. | 6, 7 |
| FiLM | Feature-wise Linear Modulation | Conditioning by predicting a per-channel scale and shift applied to feature maps. | 6 |
| adaLN | Adaptive Layer Normalization | FiLM-style modulation applied to LayerNorm parameters; adaLN-Zero initializes it to the identity for stable training. | 6, 10 |
| adaGN | Adaptive GroupNorm | The GroupNorm analogue of adaLN, injecting time/conditioning as a scale-shift. | 6 |
| GN | Group Normalization | Normalization over groups of channels; the default in diffusion U-Nets because it is batch-size independent. | 6 |
| BN | Batch Normalization | Normalization over the batch; generally avoided in diffusion because its statistics interact badly with noise levels. | 6, 7 |
| SiLU | Sigmoid Linear Unit (a.k.a. Swish) | The smooth activation $x\,\sigma(x)$ used throughout modern diffusion backbones. | 6 |
| CFG | Classifier-Free Guidance | Steering generation toward a condition by extrapolating between conditional and unconditional noise predictions, with no separate classifier. | 8, 9, 10 |
| CLIP | Contrastive Language–Image Pre-training | A jointly trained text/image encoder; a common source of text embeddings for conditioning. | 10 |
| T5 | Text-to-Text Transfer Transformer | A large language model whose text encoder is used to condition image generation. | 10 |
| VQ | Vector Quantization | Discrete-codebook compression of the autoencoder latent in some latent-diffusion variants. | 10 |
| KL-VAE | KL-regularized Variational Autoencoder | The continuous-latent autoencoder (with a light KL penalty) used as the front end in latent diffusion. | 10 |
Loss weighting & schedules#
| Acronym | Full form | What it means | Lesson(s) |
|---|
| P2 | Perception Prioritized (weighting) | A loss weighting that emphasizes the noise levels most relevant to perceptual quality. | 5 |
| Min-SNR-γ | Minimum Signal-to-Noise Ratio (gamma) | A weighting that clips each step’s effective SNR at $\gamma$, balancing easy and hard noise levels for faster convergence. | 5, 7 |
Optimization & engineering#
| Acronym | Full form | What it means | Lesson(s) |
|---|
| EMA | Exponential Moving Average | A slowly averaged copy of the weights used at sampling time; almost always improves diffusion sample quality. | 1, 7, 8, 9 |
| AdamW | Adam with decoupled Weight decay | The default optimizer for diffusion training. | 7 |
| WD | Weight Decay | $L_2$-style regularization on the weights (decoupled in AdamW). | 7 |
| AMP | Automatic Mixed Precision | Training/inference in mixed 16/32-bit precision for speed and memory savings. | 7, 8 |
| i.i.d. | independent and identically distributed | The standard assumption that data points are drawn independently from the same distribution. | 7 |
| GPU | Graphics Processing Unit | The accelerator hardware diffusion models are trained and sampled on. | 10 |
Distillation & fast generation#
| Acronym | Full form | What it means | Lesson(s) |
|---|
| SD / SD3 | Stable Diffusion / Stable Diffusion 3 | The widely used open latent-diffusion text-to-image model (and its third-generation, flow-matching version). | 10 |
| ADD | Adversarial Diffusion Distillation | A distillation method combining a diffusion objective with an adversarial loss for few-step generation. | 10 |
| LCM | Latent Consistency Model | A consistency-distillation approach enabling 1–4 step sampling in latent space. | 10 |
Venues & organizations (references only)#
These appear only in citations and are listed for completeness: ICML (International Conference on Machine Learning), NeurIPS (Neural Information Processing Systems), ICLR (International Conference on Learning Representations), CVPR (Computer Vision and Pattern Recognition), ICCV (International Conference on Computer Vision), AAAI (Association for the Advancement of Artificial Intelligence), JASA (Journal of the American Statistical Association), MICCAI (Medical Image Computing and Computer-Assisted Intervention), and ACM (Association for Computing Machinery).
Named terms that are not acronyms. A few load-bearing names look like jargon but are people or concepts, not abbreviations: Tweedie’s formula, Feller’s condition, Anderson’s reverse-time SDE, Jensen’s inequality, Bayes’ rule, Fokker–Planck equation, Euler–Maruyama and Heun integrators, Langevin dynamics, Lipschitz continuity, and slerp (spherical linear interpolation). They are explained where they appear in the lessons.
Alphabetical quick-index#
AAAI · ACM · adaGN · adaLN · ADD · AdamW · AMP · BN · CFG · CLIP · CVPR · DDIM · DDPM · DEIS · DiT · DPM-Solver · EDM · ELBO · EMA · FID · FiLM · GAN · GN · GPU · ICCV · ICLR · ICML · i.i.d. · JASA · KL · KL-VAE · LCM · LDM · LPIPS · MICCAI · Min-SNR-γ · MLE · MLP · MSE · NCSN · NeurIPS · NLL · ODE · P2 · RK · SD/SD3 · SDE · SiLU · SNR · T5 · U-Net · VAE · VDM · VLB · VP · VP-SDE · VQ · WD
← Course Index
· Appendix B: Mathematical Formulas →