Appendix B — Mathematical Formulas
Every key equation from the course, collected in one place and grouped by lesson. Each formula has a one-line reminder of what it says; follow the lesson link for the full derivation.
Read top to bottom, the equations retrace the whole arc: the model is defined as a function (Lesson 1), built up layer by layer (Lessons 2–5), scored against a target (Lessons 6), and improved by gradient descent and its refinements (Lessons 7–9) before being run to generate text (Lesson 10). Used as a reference, it is a quick way to recover a formula’s meaning and find where it was derived. Each symbol here matches its lesson exactly; nothing is restated in a simpler form.
Equations render with any LaTeX-aware Markdown viewer (VS Code + Markdown+Math, Obsidian, or GitHub).
← Course index · Appendix A — Acronyms · Appendix C — Attention Variants
Lesson 1 — The Big Picture
The model as a function — maps a token sequence and parameters to a next-token distribution:
$$ f:\ \mathcal V^{\le n}\times\Theta \to \Delta^{V-1},\qquad \hat{\mathbf p}=f(x;\theta). $$Autoregressive factorization — a whole sequence is scored by the chain rule of probability:
$$ P_\theta(x_1,\dots,x_n)=\prod_{t=1}^{n} P_\theta(x_t\mid x_{\lt t}),\qquad x_{\lt t}:=(x_1,\dots,x_{t-1}). $$Empirical risk minimization — training picks the parameters that minimize expected loss:
$$ \theta^\star=\arg\min_{\theta\in\mathbb R^P}\ \mathcal L(\theta),\qquad \mathcal L(\theta)=\mathbb E_{x\sim\mathcal D}\!\left[-\sum_{t} \log P_\theta(x_t\mid x_{\lt t})\right]. $$Pretraining compute — the $6PD$ rule of thumb (vs. $\approx 2P$ FLOPs/token at inference):
$$ C \approx 6\,P\,D \ \text{FLOPs}. $$Lesson 2 — Tokenization & Embeddings
Embedding matrix and lookup — one learned row per token; a lookup is a one-hot matrix product:
$$ E\in\mathbb R^{V\times d},\qquad \mathbf x_i = \mathbf e_i^\top E \in\mathbb R^{d}. $$Embedding gradient — only the rows of tokens that actually appeared get updated (sparse):
$$ \frac{\partial\mathcal L}{\partial E_{j,:}}=\sum_{\text{positions }i\,:\,x_i=j}\frac{\partial\mathcal L}{\partial \mathbf x_i}. $$Sinusoidal positional encoding — fixed sine/cosine features at geometrically spaced frequencies:
$$ PE_{(p,2k)}=\sin\!\big(p\,/\,10000^{2k/d}\big),\qquad PE_{(p,2k+1)}=\cos\!\big(p\,/\,10000^{2k/d}\big). $$RoPE — rotate each query/key pair by a position-dependent angle:
$$ \begin{pmatrix}q'_{2k}\\ q'_{2k+1}\end{pmatrix} =\begin{pmatrix}\cos p\omega_k & -\sin p\omega_k\\ \sin p\omega_k & \cos p\omega_k\end{pmatrix} \begin{pmatrix}q_{2k}\\ q_{2k+1}\end{pmatrix}. $$Lesson 3 — Self-Attention
Query/key/value projections:
$$ Q=HW_Q\in\mathbb R^{n\times d_k},\quad K=HW_K\in\mathbb R^{n\times d_k},\quad V=HW_V\in\mathbb R^{n\times d_v}. $$Scaled dot-product attention — the core operation (with mask $M$):
$$ \mathrm{Attn}(Q,K,V)=\mathrm{softmax}\!\Big(\tfrac{QK^\top}{\sqrt{d_k}}+M\Big)\,V. $$Per-token view — output is a softmax-weighted average of value vectors:
$$ \mathbf o_i=\sum_{j=1}^{n}A_{ij}\,\mathbf v_j,\qquad A_{ij}=\frac{\exp(s_{ij})}{\sum_{m}\exp(s_{im})},\qquad s_{ij}=\frac{\mathbf q_i^\top\mathbf k_j}{\sqrt{d_k}}. $$The $\sqrt{d_k}$ rationale — raw scores have variance $d_k$, so we divide to keep softmax well-scaled:
$$ \mathbf q_i^\top\mathbf k_j=\sum_{l=1}^{d_k} q_{il}k_{jl},\qquad \mathbb E[\cdot]=0,\qquad \mathrm{Var}[\cdot]=d_k. $$Softmax Jacobian — governs gradient flow through attention weights:
$$ \frac{\partial a_i}{\partial s_j}=a_i(\delta_{ij}-a_j) \;\Longleftrightarrow\; J=\mathrm{diag}(\mathbf a)-\mathbf a\mathbf a^\top. $$Causal mask — blocks attention to future positions:
$$ M_{ij}=\begin{cases}0,& j\le i\\[2pt] -\infty,& j\gt i.\end{cases} $$Multi-head attention — concatenate independent heads and project:
$$ \mathrm{MHA}(H)=\big[\,\mathrm{head}_1\,\|\cdots\|\,\mathrm{head}_h\,\big]W_O,\quad \mathrm{head}_i=\mathrm{Attn}(HW_Q^{(i)},HW_K^{(i)},HW_V^{(i)}). $$Lesson 4 — The Transformer Block
Pre-norm residual block — normalize, sublayer, add; twice per block:
$$ \tilde H = H + \mathrm{MHA}\big(\mathrm{Norm}(H)\big),\quad H' = \tilde H + \mathrm{MLP}\big(\mathrm{Norm}(\tilde H)\big). $$Position-wise MLP — up-project, nonlinearity, down-project:
$$ \mathrm{MLP}(\mathbf x)=W_2\,\phi\!\big(W_1\mathbf x+\mathbf b_1\big)+\mathbf b_2,\qquad W_1\in\mathbb R^{d_{ff}\times d},\ W_2\in\mathbb R^{d\times d_{ff}}. $$Residual Jacobian — the identity term is why deep stacks still train:
$$ \frac{\partial \mathbf y}{\partial \mathbf x}=I+\frac{\partial\mathcal F}{\partial\mathbf x}. $$LayerNorm (forward) — standardize per token, then learned affine:
$$ \mu=\tfrac1d\textstyle\sum_k x_k,\quad \sigma^2=\tfrac1d\sum_k(x_k-\mu)^2,\quad \mathrm{LN}(\mathbf x)=\gamma\odot\frac{\mathbf x-\mu}{\sqrt{\sigma^2+\epsilon}}+\beta. $$LayerNorm (backward) — gradient projects out the mean and normalized-direction:
$$ \frac{\partial\mathcal L}{\partial x_i}=\frac{1}{\sqrt{\sigma^2+\epsilon}}\Big(g_i-\tfrac1d\textstyle\sum_k g_k-\hat x_i\cdot\tfrac1d\sum_k g_k\hat x_k\Big). $$RMSNorm — divide by root-mean-square, no mean subtraction:
$$ \mathrm{RMS}(\mathbf x)=\frac{\mathbf x}{\sqrt{\tfrac1d\sum_k x_k^2+\epsilon}}\odot\gamma. $$Lesson 5 — Output Head & Objective
Logits and softmax — project hidden state to vocabulary scores, then normalize:
$$ \mathbf z_i=\mathbf h_i^\top W_U\in\mathbb R^V,\qquad \hat p_w=\frac{e^{z_w}}{\sum_{m=1}^{V}e^{z_m}}. $$Sequence log-likelihood and cross-entropy loss:
$$ \log P_\theta(x_{1:n})=\sum_{t=1}^{n}\log \hat p_{\,t,\,x_t},\qquad \mathcal L(\theta)=-\frac1n\sum_{t=1}^n\log \hat p_{\,t,\,x_t}. $$The clean logit-gradient identity — predicted minus target:
$$ \frac{\partial\mathcal L}{\partial\mathbf z}=\hat{\mathbf p}-\mathbf y. $$The full parameter set:
$$ \theta=\Big\{E,\ \{W_Q,W_K,W_V,W_O\}_{\ell=1}^N,\ \{W_1,W_2,\mathbf b_1,\mathbf b_2\}_{\ell=1}^N,\ \{\gamma,\beta\}_{\ell},\ W_U\Big\}. $$Lesson 6 — The Loss Function
Cross-entropy and its decomposition — entropy plus KL divergence:
$$ H(p,\hat p)=-\sum_{w=1}^{V}p_w\log\hat p_w =\underbrace{-\sum_w p_w\log p_w}_{H(p)}\;+\;\underbrace{\sum_w p_w\log\frac{p_w}{\hat p_w}}_{D_{\mathrm{KL}}(p\,\|\,\hat p)}. $$Hence minimizing loss is minimizing KL to the true distribution:
$$ \min_\theta H(p,\hat p_\theta)\iff \min_\theta D_{\mathrm{KL}}(p\,\|\,\hat p_\theta). $$With one-hot targets, the loss collapses to negative log-probability of the seen token:
$$ H(\mathbf y,\hat{\mathbf p})=-\log\hat p_{x_t}. $$Perplexity — the exponential of the loss:
$$ \mathrm{PPL}=\exp\!\big(\mathcal L\big)=\exp\!\Big(-\tfrac1n\textstyle\sum_t\log\hat p_{t,x_t}\Big). $$Lesson 7 — Backpropagation & Gradient Descent
Gradient vector and the gradient-descent update:
$$ \nabla_\theta\mathcal L=\Big(\tfrac{\partial\mathcal L}{\partial\theta_1},\dots,\tfrac{\partial\mathcal L}{\partial\theta_P}\Big),\qquad \theta\leftarrow\theta-\eta\,\nabla_\theta\mathcal L. $$Vector-Jacobian products — the backprop primitive that propagates adjoints layer by layer:
$$ \bar{\mathbf a}^{(\ell-1)}=\Big(\frac{\partial g_\ell}{\partial \mathbf a^{(\ell-1)}}\Big)^{\!\top}\bar{\mathbf a}^{(\ell)}, \qquad \bar\theta_\ell=\Big(\frac{\partial g_\ell}{\partial \theta_\ell}\Big)^{\!\top}\bar{\mathbf a}^{(\ell)}. $$Linear-layer adjoints — the VJPs for a matrix multiply:
$$ \bar W=\bar{\mathbf u}\,\mathbf x^\top,\qquad \bar{\mathbf x}=W^\top\bar{\mathbf u}. $$Lesson 8 — Optimizers, Learning Rate & Batches
Momentum — step along an EMA of gradients:
$$ \mathbf m_t=\beta\,\mathbf m_{t-1}+(1-\beta)g_t,\qquad \theta\leftarrow\theta-\eta\,\mathbf m_t. $$Adam moments and their bias correction:
$$ \mathbf m_t=\beta_1\mathbf m_{t-1}+(1-\beta_1)g_t,\qquad \mathbf v_t=\beta_2\mathbf v_{t-1}+(1-\beta_2)g_t^{\odot2}, $$$$ \hat{\mathbf m}_t=\frac{\mathbf m_t}{1-\beta_1^{\,t}},\qquad \hat{\mathbf v}_t=\frac{\mathbf v_t}{1-\beta_2^{\,t}}. $$Adam update and AdamW (decoupled weight decay):
$$ \theta\leftarrow\theta-\eta\,\frac{\hat{\mathbf m}_t}{\sqrt{\hat{\mathbf v}_t}+\epsilon}, \qquad \theta\leftarrow\theta-\eta\Big(\frac{\hat{\mathbf m}_t}{\sqrt{\hat{\mathbf v}_t}+\epsilon}+\lambda\theta\Big). $$Cosine learning-rate schedule:
$$ \eta_t=\eta_{\min}+\tfrac12(\eta_{\max}-\eta_{\min})\Big(1+\cos\tfrac{\pi(t-T_w)}{T-T_w}\Big). $$Scaling laws — power-law loss in parameters and data, above an irreducible floor:
$$ \mathcal L(N_{\text{params}},D)\approx \mathcal L_\infty + \Big(\tfrac{A}{N_{\text{params}}}\Big)^{\alpha}+\Big(\tfrac{B}{D}\Big)^{\beta}. $$Lesson 9 — Fine-Tuning, SFT & RLHF
SFT loss with response masking — train only on response tokens:
$$ \mathcal L_{\text{SFT}}(\theta)=-\sum_{t}\,\mathbb 1[t\in\text{response}]\;\log P_\theta\big(y_t\mid x,\,y_{\lt t}\big). $$LoRA — a low-rank correction to frozen weights:
$$ W=W_0+\Delta W,\qquad \Delta W=\frac{\alpha}{r}\,BA,\quad A\in\mathbb R^{r\times k},\ B\in\mathbb R^{d\times r}. $$Reward-model loss (Bradley–Terry) over preference pairs:
$$ \mathcal L_{\text{RM}}(\phi)=-\,\mathbb E_{(x,y_w,y_l)}\Big[\log\sigma\big(r_\phi(x,y_w)-r_\phi(x,y_l)\big)\Big]. $$KL-anchored RL objective and the PPO clipped surrogate:
$$ \max_\theta\ \mathbb E_{x\sim\mathcal D,\,y\sim\pi_\theta(\cdot\mid x)}\Big[\,r_\phi(x,y)\;-\;\beta\,\log\frac{\pi_\theta(y\mid x)}{\pi_{\text{ref}}(y\mid x)}\Big], $$$$ \mathcal L^{\text{CLIP}}(\theta)=\mathbb E_t\Big[\min\big(\rho_t\hat A_t,\ \mathrm{clip}(\rho_t,1-\epsilon,1+\epsilon)\,\hat A_t\big)\Big]. $$DPO loss — optimize preferences directly, no separate reward model:
$$ \mathcal L_{\text{DPO}}(\theta)=-\,\mathbb E_{(x,y_w,y_l)}\Big[\log\sigma\Big(\beta\log\tfrac{\pi_\theta(y_w\mid x)}{\pi_{\text{ref}}(y_w\mid x)}-\beta\log\tfrac{\pi_\theta(y_l\mid x)}{\pi_{\text{ref}}(y_l\mid x)}\Big)\Big]. $$Lesson 10 — Inference in Depth
Generation distribution — same autoregressive product, now conditioned on the prompt:
$$ P_\theta(y_{1:m}\mid x)=\prod_t P_\theta(y_t\mid x,y_{\lt t}). $$Temperature scaling — rescale logits to sharpen or flatten the distribution:
$$ \hat p_w(T)=\frac{e^{z_w/T}}{\sum_m e^{z_m/T}}. $$Top-p (nucleus) sampling — smallest token set whose mass reaches $p$:
$$ S=\arg\min\big\{|S|:\textstyle\sum_{w\in S}\hat p_w\ge p\big\}. $$KV-cache memory — linear in sequence length, depth, and width ($\times 2$ for K and V):
$$ \text{bytes}=2\times n\times N_{\text{layers}}\times d\times (\text{bytes/elem}). $$← Course index · Appendix A — Acronyms · Appendix C — Attention Variants