Chapter 1 — What These Machines Actually Are
Before we open the hood on any particular model, it helps to settle what kind of thing we are looking at. The systems in this book go by intimidating names and produce uncanny results, but underneath they are far simpler in concept than they appear. This chapter lays down the three ideas that the rest of the book will lean on constantly. None of them requires any mathematics. All of them are easy to misunderstand, which is precisely why we start here.
A model is just a function
Strip away the branding and an AI model is a function: something that takes an input and produces an output. You put text in; you get text out. You put a prompt in; you get an image out. That is the whole shape of it. What makes these particular functions remarkable is not the idea of input-and-output — your pocket calculator does that — but the sheer richness of what they have absorbed about the world in order to produce sensible outputs.
The richness lives in what we will call the model’s settings: an enormous collection of internal numbers, often numbering in the billions, that determine exactly how an input gets transformed into an output. People call these numbers parameters. You can think of them as billions of tiny knobs. Turn the knobs one way and the function behaves one way; turn them another way and it behaves differently. A “trained model” is nothing more than one very particular setting of all those knobs — a setting that happens to produce useful, coherent results.
This already tells us something important. The model has no secret database it consults, no little library it flips through. Everything it “knows” is baked into the positions of those knobs. When a language model tells you the capital of France, it is not looking it up. The fact that Paris is the answer has been pressed into the arrangement of its knobs, the same way a song is pressed into the grooves of a record.
The most important distinction in the book: training versus using
Here is the single idea that clears up more confusion than any other. There are two completely separate phases in the life of one of these models, and they could not be more different.
The first phase is training. This is how the model is built. Over weeks or months, on vast banks of specialized computers, the model is shown staggering quantities of material — for a language model, trillions of words; for an image model, billions of pictures — and its billions of knobs are slowly, patiently adjusted until its outputs become good. Training is the only time the model changes. It is expensive, it is slow, and it happens once.
The second phase is using the model — the technical word is inference. This is what happens when you type a question and get an answer. And here is the crucial part: during this phase, the model does not change at all. Its knobs are frozen solid. It reads your input, runs it through the fixed function, and produces an output. Then it does it again for the next input. Nothing about the model itself is altered, no matter how long or how cleverly you talk to it.
There is a phrase worth memorizing: training writes; using reads. Building the model writes the knobs into place. Using the model only reads them.
This distinction explains a question almost everyone asks: does the model learn from our conversation? The answer is no. The reason a chatbot seems to remember what you said three paragraphs ago is not that it learned anything — it is simply that your earlier words are still sitting right there in front of it as part of the input, the way a person re-reading the top of a page is not someone whose brain rewired itself mid-sentence. When the conversation ends and a new one begins, that context is gone, and the model is exactly what it was before. It never learned you; it only ever re-read the page.
This also explains the wild difference in cost between the two phases. Building one of these models can cost a fortune and consume the output of a small power plant. Running it, by comparison, is cheap — cheap enough that a company can train a model once and then serve it to millions of people. The hard, costly work is front-loaded into a single act of construction.
Figure 1. Training writes; using reads.
Three kinds of numbers, never to be confused
Inside any of these systems there are numbers everywhere, and they are not all the same kind of thing. Sorting them into three buckets is one of the most clarifying habits you can develop, and the original courses tag every number this way for exactly that reason.
The first bucket is the learned numbers — the knobs, the parameters. These are adjusted by training and by nothing else. They are the model. When someone says “a model with seventy billion parameters,” they are counting the knobs in this bucket.
The second bucket is the chosen numbers — the settings a human picks before or during training, and which the model never adjusts on its own. How many knobs the model should have, how big to make it, how adventurous to let it be when it answers — these are human decisions. They shape the model and the process, but they are not learned; they are dialed in.
The third bucket is the scratch numbers — temporary working values that exist only during a single run and are thrown away the instant it finishes. When the model processes your sentence, it generates a great deal of intermediate arithmetic to get from input to output. None of it is kept. It is the mental scratch paper the model covers in pencil and then discards.
Keep these three apart and a surprising amount of mystery evaporates. “Did the model learn that?” becomes a precise question: did it change a knob in the first bucket (no, not while you were talking to it), or are you seeing the effect of a human setting in the second bucket, or just scratch work in the third? Throughout this book, when something seems puzzling, ask which bucket of numbers is really involved.
Figure 2. Three kinds of numbers, and how to tell them apart.
Why “guess the next thing” turns out to be enough
There is one more foundational idea, and it is the one that makes everything else possible. Most of the models in this book are trained on a goal that sounds almost insultingly modest: predict what comes next.
For a language model, “what comes next” is the next word. You show it a stretch of text with the next word hidden, and you train it to guess that word. That is the entire training game, repeated across an unfathomable amount of text. It seems impossible that such a narrow exercise could give rise to essays, working code, translation, and argument. The resolution to that paradox is the intellectual heart of the whole field, so it is worth stating carefully.
To guess the next word well, across every imaginable sentence and topic, a model is quietly forced to learn far more than word-frequency trivia. To finish “the boiling point of water at sea level is one hundred degrees ___” it has to have absorbed some physics. To finish “she opened the door and gasped, because standing there was ___” it has to track narrative and surprise. To finish a half-written function it has to grasp the logic of code. Predicting the next word turns out to be a back door into modeling the whole structure of language — and, through language, a great deal of the structure of the world. The modest goal is a Trojan horse.
This is why you will see the phrase “next-token prediction” treated with such reverence. It is not a party trick bolted onto something cleverer. It is the cleverness. (Image models, as we will see in Chapter 8, are trained on a different but equally humble goal — “guess the noise” — with the same surprising payoff.)
Figure 3. What guessing the next word quietly forces a model to learn.
What to carry forward
We now have the frame. A model is a function whose behavior is set by billions of learned knobs. It is built once, in an expensive training phase that writes those knobs, and then used in a frozen state that only reads them — which is why it does not learn from your conversation. Its numbers come in three kinds: learned, chosen, and scratch. And the engine that drives most of it is the unreasonably powerful goal of predicting what comes next.
With that settled, we can start opening the box. The first thing we need to understand is how something as fuzzy and human as language gets turned into something a machine can compute with at all. That is the subject of Chapter 2 .
Contents · Chapter 2 · From Words to Numbers →
The figures in this lesson follow the shared figure key , which explains what every shape, line style and colour means.
