Chapter 12 — The Whole Landscape
We have come a long way from “a model is just a function.” We have opened the box on five different kinds of system and walked the full arc from a pile of random knobs to a deployed, helpful assistant. This closing chapter does something the individual chapters could not: it steps back far enough to see the whole landscape at once — how the pieces relate, what they share, where they differ, and how to think about which one suits a given job. If you read nothing else after the foundations, read this.
One toolkit, rearranged
The single most important thing to carry away is how little there really is at the bottom, and how much comes from rearranging it. Across the entire family of language models — the storyteller, the reader, the translator, the committee — the underlying parts are the same: text becomes points on a map of meaning; attention lets those points exchange information; the result rises through a tall tower of blocks, kept stable by shortcut connections and normalization; and learning happens by the same downhill walk, scoring wrongness and nudging knobs.
What distinguishes the models is mostly what each word is allowed to look at, and what job the whole thing is pointed at. Let a word see only what came before, and train it to predict the next word, and you have a storyteller that writes. Let every word see in both directions, and train it to fill in blanks, and you have a reader that understands but cannot write. Give reading and writing their own stacks, bonded by cross-attention, and you have a translator that reads one thing and writes another. Replace the per-word processing stage with a bank of experts and a router, and you have a committee that knows enormously while working cheaply. Four famous, distinct-seeming systems, all built from one toolkit with the dials set differently.
Diffusion stands a little apart, because it generates images by refining a whole noisy canvas rather than producing tokens one at a time — but even it reuses the same deep ideas, learning by minimizing an error and steering itself with the very same cross-attention to follow a text prompt. The unity runs deeper than the surface variety suggests.
Figure 1. Four shapes, one toolkit, the dials set differently.
The cast, in one line each
It helps to have the whole cast in view at once.
A decoder model (GPT, Llama, Claude) is a writer. It generates text left to right, one word at a time, and is the all-purpose workhorse behind chatbots, coding assistants, and open-ended generation. When people say “large language model,” this is usually what they mean.
An encoder model (the BERT family) is a reader. It takes in text bidirectionally and produces a deep understanding of it, but cannot generate. It is the quiet, cheap, high-throughput engine behind search, classification, content moderation, and the retrieval that feeds many larger systems.
An encoder–decoder model (T5, BART) is a translator. It bolts a reader to a writer to transform one text into another, and it shines at translation and summarization, where the input must be understood completely before the output is composed.
A mixture-of-experts model (Switch, Mixtral, DeepSeek) is not a separate task but a separate strategy: a way to build any of the above — usually a writer — so that it holds vast knowledge while only activating a small part of itself per word. It trades cheap memory for the quality of a far larger model, at the price of a more delicate training process and a heavier memory footprint when run.
A diffusion model (Stable Diffusion, DALL·E) is a sculptor. It generates images, audio, video, and more by starting from pure noise and repeatedly refining the whole thing toward a coherent result, guided by a prompt.
Figure 2. The cast, in one line each.
Two philosophies of generation
Among the generators, there is a clean philosophical split worth naming, because it captures a real divide in how machines create.
The language models generate sequentially: one piece at a time, in order, each piece committed before the next, the output built up like a sentence written word by word and never revised. This suits language, where order is the essence of meaning and each word genuinely depends on the ones before it.
The diffusion models generate all at once, then refine: the entire output is present from the start as noise, and the whole thing is sharpened together over many passes, like a photograph developing. This suits images, where everything must be globally coherent at the same time — the two eyes matching, the light consistent across the frame — which is hard to guarantee if you commit to one piece at a time in some arbitrary order.
Neither is more advanced than the other. They are matched to the shape of what they make. And the boundaries are blurring — researchers have built image generators that work sequentially and language generators that refine all at once — but the two instincts remain the clearest way to organize the zoo of generative models.
How to think about choosing
If you ever find yourself deciding which of these to reach for, a few plain questions cut through most of the confusion.
Do you need to generate free-form text, hold a conversation, or write code? That is a writer’s job — a decoder model. If you also have no training data and just want it to follow instructions, lean on its raised, instruction-following form and a good prompt.
Do you need to classify, tag, score, extract, or search through text you already have, cheaply and at scale? That is a reader’s job — an encoder model — and its clean, bounded, trustworthy outputs are a real advantage for high-volume or sensitive work.
Do you need to turn one whole text into a different one — translate it, summarize it? That is the translator’s sweet spot, an encoder–decoder, though in practice a strong writer with a good prompt often does the job too.
Do you want the quality of a giant model but cannot afford to run a giant model’s worth of computation on every word? That is the case for a mixture-of-experts design — provided you can afford the memory to keep all those experts loaded.
Do you need to create images or other rich media from a description? That is diffusion.
And the most important real-world answer is often more than one of them, cooperating. The dominant pattern in applied AI today has a cheap reader retrieve the relevant documents and a capable writer compose an answer from them — each model doing exactly what its shape was built for. These systems are far more often partners than rivals.
Figure 3. Five plain questions that cut through most of the confusion.
The two ideas that hold it all together
We opened the book with three foundational ideas. We close by reaffirming the two that, more than any others, will keep you oriented no matter how the field changes.
The first is training writes; using reads. A model is built once, in an expensive act of construction that sets its knobs, and then frozen. When you use it, it is not learning — it is reading. Almost every popular misconception about these systems traces back to forgetting this.
The second is learned, chosen, and scratch. Some numbers are the model itself, written by training. Some are settings a human dialed in. Some are mere working values, discarded after each run. Keep these three apart and the fog clears: questions like “did it learn that?” or “where does its behavior come from?” become answerable rather than mystifying.
Everything else in this book — attention, the towers of blocks, the readers and writers and committees and sculptors, the downhill walk of learning, the raising of an assistant, the dials of generation — is built on top of those two ideas. The field will keep producing new names and new records, and the specific architectures will keep shifting. But the shape of the thing, the grammar of how these machines are built and what they do, is what you now understand. New developments will mostly be variations on the themes in these chapters: a different rule for what each word may see, a cleverer way to spend compute or memory, a new game to learn from, a faster way to generate. You have the map. The territory will keep changing, but the map will keep working.
For the meaning of any term or acronym used along the way, the Glossary collects them all in plain language. Thank you for reading.
← Chapter 11 · How a Model Thinks in Real Time · Contents · Appendix A · A Field Guide to Real Models →
The figures in this lesson follow the shared figure key , which explains what every shape, line style and colour means.
