Appendix B — The Encoder vs. Decoder Cheat Sheet
Technical course on encoder LLM mechanics — Appendix to all lessons.
Where this fits
This appendix collects every contrast drawn across the ten lessons into one place, organized by where in the stack the difference lives. Use it as a lookup table when you need the short answer, with a pointer back to the lesson that derives it. The single root cause to keep in mind: a decoder is causal and generates the next token; an encoder is bidirectional and emits representations. Every row below is a consequence of that one difference (Lesson 1).
Figure 1. One deleted mask, and everything that follows from it.
The master table
| Aspect | Decoder (GPT/Llama-style) | Encoder (BERT-style) | Lesson |
|---|---|---|---|
| Core object | $f(x;\theta)\to\Delta^{V-1}$ (next-token distribution) | $g(x;\theta)\to\mathbb R^{n\times d}$ (contextual vectors) | 1 |
| Direction | causal (left-to-right) | bidirectional (both sides) | 1, 3 |
| Factorization | autoregressive $\prod_t P(x_t\mid x_{\lt t})$ | none (no valid joint) | 1, 7 |
| Purpose | generate text | understand / represent text | 1 |
| Tokenizer | typically BPE | typically WordPiece (## marks) | 2 |
| Input embedding | token + position | token + segment + position | 2 |
| Special tokens | minimal (chat markers) | [CLS], [SEP], [MASK], [PAD] | 2 |
| Position scheme | RoPE / ALiBi (extrapolates) | learned-absolute, ≤512 (ModernBERT: RoPE) | 2 |
| Attention mask | causal + padding | padding only (no causal) | 3 |
| Attention matrix | lower-triangular | full $n\times n$ | 3 |
| Block norm | trend pre-LN | original post-LN | 4 |
| Privileged position | last token (sees whole prefix) | [CLS], position 0 (by convention) | 4 |
| Pretraining objective | next-token prediction | masked language modeling (denoising) | 5 |
| Signal density | 100% of positions scored | ~15% of positions scored | 5, 8 |
| Coherence signal | implicit in next-token | NSP (dropped) / SOP | 5 |
| Output head | one vocabulary projection, everywhere | typed heads per task (pooled / per-token / span) | 6 |
| Sequence summary | last hidden state (automatic) | pooling choice ([CLS] / mean) | 6 |
| Intrinsic metric | perplexity (well-defined) | undefined; pseudo-perplexity stand-in | 7 |
| Primary evaluation | zero-/few-shot generation quality | GLUE/SQuAD/MTEB after fine-tuning | 7 |
| Inference shape | $n$ sequential passes + KV cache | 1 parallel pass, no KV cache | 1, 3 |
| Adaptation | SFT / RLHF / DPO / prompting | attach typed head + supervised FT | 9 |
| Needs labeled data to adapt? | often no (few-shot) | yes | 9 |
| Output type | sampled text | structured label / tags / span / vector | 6, 9 |
| Can it generate? | yes (its whole point) | no | 1, 3 |
Figure 2. The master contrast, selected rows.
The one-paragraph summary
A decoder hides the future with a causal mask so it can be trained to predict the next token, which makes it a generator: it factorizes sequences autoregressively, runs an $n$-step KV-cached loop at inference, routes every task through one vocabulary head, is measured by perplexity, and is adapted by alignment or prompting. An encoder deletes that mask, so every token sees the whole sequence; this kills next-token training (the answer is visible), forcing the masked-language-modeling denoising objective that scores only ~15% of positions; it emits a matrix of contextual representations rather than a distribution, so it needs typed heads and pooling to make predictions, has no perplexity (no valid joint), runs in a single parallel pass with no KV cache, and is adapted by supervised fine-tuning on labeled data. Decoders generate; encoders understand; encoder–decoders transform; and real systems (RAG) use them together.
Decision rule, compressed
produce new text?"} Q -->|yes| G{"Transform a long
given input into output?"} Q -->|"no — classify, tag,
extract, or embed"| E["ENCODER
(single pass, typed head)"] G -->|yes| ED["ENCODER–DECODER
(T5 / BART)"] G -->|"no — free-form
chat / completion"| D["DECODER
(autoregressive)"] E --> RAG{"Building search
or RAG?"} RAG -->|yes| BOTH["BOTH:
encoder retrieves,
decoder generates"]
Figure 3. The decision rule, compressed.
← Course index · Extends every lesson
Appendices: A — Encoder Family · B — Encoder vs. Decoder Cheat Sheet · C — Acronyms
The figures in this lesson follow the shared figure key , which explains what every shape, line style and colour means.
