https://www.fanfani.net/tutorials/inside-the-machines/Recent content in Inside the Machines on Ship of TheseusHugo -- 0.152.2en-usSun, 23 Aug 2026 00:00:00 +0000https://www.fanfani.net/tutorials/inside-the-machines/chapter-01-what-these-machines-are/Mon, 01 Jan 0001 00:00:00 +0000https://www.fanfani.net/tutorials/inside-the-machines/chapter-01-what-these-machines-are/<p>Chapter 1 — What These Machines Actually Are Before we open the hood on any particular model, it helps to settle what kind of thing we are looking at. The systems in this book go by intimidating names and produce uncanny results, but underneath they are far simpler in concept than they appear. This chapter lays down the three ideas that the rest of the book will lean on constantly. None of them requires any mathematics. All of them are easy to misunderstand, which is precisely why we start here.</p><p><a href="https://www.fanfani.net/tutorials/inside-the-machines/chapter-01-what-these-machines-are/">Read the full post →</a></p>https://www.fanfani.net/tutorials/inside-the-machines/chapter-02-from-words-to-numbers/Mon, 01 Jan 0001 00:00:00 +0000https://www.fanfani.net/tutorials/inside-the-machines/chapter-02-from-words-to-numbers/<p>Chapter 2 — From Words to Numbers In the last chapter we settled what a model is: a function, set by billions of learned knobs, that turns an input into an output. But there is a gap in that picture we glossed over. A computer, at bottom, only ever does arithmetic. It manipulates numbers. And yet we hand it words — fuzzy, ambiguous, deeply human things. Something has to bridge that gap before any of the clever machinery can begin. This chapter is about that bridge. It comes in two steps: first chopping text into pieces, and then turning each piece into a point on a vast &ldquo;map of meaning.&rdquo;</p><p><a href="https://www.fanfani.net/tutorials/inside-the-machines/chapter-02-from-words-to-numbers/">Read the full post →</a></p>https://www.fanfani.net/tutorials/inside-the-machines/chapter-03-attention/Mon, 01 Jan 0001 00:00:00 +0000https://www.fanfani.net/tutorials/inside-the-machines/chapter-03-attention/<p>Chapter 3 — Attention: The Idea That Changed Everything At the end of the last chapter we left every word stranded on its own island. Each one had been placed on the map of meaning and stamped with its position, but each one&rsquo;s meaning was still computed in isolation. The word &ldquo;it&rdquo; had no idea what it stood for. The word &ldquo;bank&rdquo; could not tell a river from a vault. To understand anything, the words must look at one another and revise themselves accordingly. The mechanism that lets them do this is called attention, and it is the breakthrough that made the modern era of AI possible. Almost every system in this book is built on it. So it is worth slowing down and getting the intuition exactly right.</p><p><a href="https://www.fanfani.net/tutorials/inside-the-machines/chapter-03-attention/">Read the full post →</a></p>https://www.fanfani.net/tutorials/inside-the-machines/chapter-04-decoder-models/Mon, 01 Jan 0001 00:00:00 +0000https://www.fanfani.net/tutorials/inside-the-machines/chapter-04-decoder-models/<p>Chapter 4 — The Storytellers: Decoder Models We now have our three foundations in hand: a model is a function set by learned knobs, text becomes points on a map of meaning, and attention lets those points talk to one another. From here on, we are simply arranging this machinery into different shapes for different jobs. The first and most famous shape is the decoder model — the kind of system behind ChatGPT, Claude, Llama, and Gemini. These are the storytellers: models built to write, one word at a time. This chapter is about how they work and what makes them so capable. (We will save the deep details of how they are trained and run for Part IV, where those topics get a full treatment across all model types. Here we focus on the architecture itself.)</p><p><a href="https://www.fanfani.net/tutorials/inside-the-machines/chapter-04-decoder-models/">Read the full post →</a></p>https://www.fanfani.net/tutorials/inside-the-machines/chapter-05-encoder-models/Mon, 01 Jan 0001 00:00:00 +0000https://www.fanfani.net/tutorials/inside-the-machines/chapter-05-encoder-models/<p>Chapter 5 — The Readers: Encoder Models The storyteller of the last chapter is built to write. But writing is not the only thing one might want from a language machine. Often you already have the text — a review, an email, a legal clause, a search query — and what you want is not more text but understanding: is this review positive or negative, is this email spam, which words here are people&rsquo;s names, does this query mean the same thing as that document. For jobs like these, the machinery from Part I is arranged into a different shape entirely. These are encoder models, and the most famous of them is BERT. If the decoder is a writer, the encoder is a reader.</p><p><a href="https://www.fanfani.net/tutorials/inside-the-machines/chapter-05-encoder-models/">Read the full post →</a></p>https://www.fanfani.net/tutorials/inside-the-machines/chapter-06-encoder-decoder-models/Mon, 01 Jan 0001 00:00:00 +0000https://www.fanfani.net/tutorials/inside-the-machines/chapter-06-encoder-decoder-models/<p>Chapter 6 — The Translators: Encoder–Decoder Models We have met the writer, who generates text from left to right, and the reader, who understands text in a single bidirectional pass. Now consider a job that seems to need both at once: translation. To translate an English sentence into German, you must first read the English completely — understanding it fully, including how its ending clarifies its beginning — and then write a fresh German sentence based on that understanding. Summarization is the same shape: read a long article thoroughly, then write a short summary. Answering a question about a passage is the same shape again. These &ldquo;read one thing, write a different thing&rdquo; tasks have a name — sequence-to-sequence — and they are the natural home of the encoder–decoder model, the family that includes T5 and BART.</p><p><a href="https://www.fanfani.net/tutorials/inside-the-machines/chapter-06-encoder-decoder-models/">Read the full post →</a></p>https://www.fanfani.net/tutorials/inside-the-machines/chapter-07-mixture-of-experts/Mon, 01 Jan 0001 00:00:00 +0000https://www.fanfani.net/tutorials/inside-the-machines/chapter-07-mixture-of-experts/<p>Chapter 7 — The Committee: Mixture-of-Experts Every model we have met so far shares a hidden assumption: when it processes a word, it uses all of itself to do it. Every one of its billions of knobs participates in every word. This is so natural that it is easy to miss as an assumption at all — but it has an expensive consequence, and the family of models in this chapter exists to escape it. These are mixture-of-experts models, the family that includes Switch, Mixtral, and DeepSeek, and their central idea is to build a model that is vast in what it knows while only switching on a small part of itself for any given word.</p><p><a href="https://www.fanfani.net/tutorials/inside-the-machines/chapter-07-mixture-of-experts/">Read the full post →</a></p>https://www.fanfani.net/tutorials/inside-the-machines/chapter-08-diffusion-models/Mon, 01 Jan 0001 00:00:00 +0000https://www.fanfani.net/tutorials/inside-the-machines/chapter-08-diffusion-models/<p>Chapter 8 — Sculpting Images from Noise: Diffusion Models Everything in Part II generated language, and all of it generated language the same way: one word at a time, left to right, each word committed before the next. This chapter leaves language behind for images, and in doing so it introduces a completely different philosophy of creation. The systems behind Stable Diffusion, DALL·E, and Midjourney do not build a picture piece by piece in some fixed order. They start with a square of pure visual static and gradually clarify the whole image at once, like a Polaroid developing out of a haze. The method is called diffusion, and once you see its central trick, it is one of the most satisfying ideas in the field.</p><p><a href="https://www.fanfani.net/tutorials/inside-the-machines/chapter-08-diffusion-models/">Read the full post →</a></p>https://www.fanfani.net/tutorials/inside-the-machines/chapter-09-how-a-model-learns/Mon, 01 Jan 0001 00:00:00 +0000https://www.fanfani.net/tutorials/inside-the-machines/chapter-09-how-a-model-learns/<p>Chapter 9 — How a Model Learns Throughout this book we have spoken of a model&rsquo;s billions of knobs being &ldquo;adjusted during training&rdquo; without ever saying how. We deferred it on purpose, because the answer is the same for every architecture in the book — the storyteller, the reader, the translator, the committee, even the image-sculptor of Chapter 8. They differ in shape and in the game they play, but the engine that turns a pile of random knobs into a capable system is one and the same. This chapter is that engine, told without a single equation. It comes down to three ideas: measuring how wrong the model is, working out which way to nudge every knob, and taking a careful step in that direction — over and over, a staggering number of times.</p><p><a href="https://www.fanfani.net/tutorials/inside-the-machines/chapter-09-how-a-model-learns/">Read the full post →</a></p>https://www.fanfani.net/tutorials/inside-the-machines/chapter-10-raising-a-helpful-assistant/Mon, 01 Jan 0001 00:00:00 +0000https://www.fanfani.net/tutorials/inside-the-machines/chapter-10-raising-a-helpful-assistant/<p>Chapter 10 — Raising a Helpful Assistant At the end of the last chapter we had a fully trained model — fluent, knowledgeable, and, surprisingly, not very useful. This chapter is about the gap between those two things, and how it gets closed. It is one of the most important and least understood stages in the whole story, because it is where a raw text-predictor becomes the polite, helpful, well-behaved assistant people actually talk to. The crucial thing to hold onto throughout is that no new kind of learning appears here. Every stage below is the same downhill walk from Chapter 9. All that changes is the teaching material.</p><p><a href="https://www.fanfani.net/tutorials/inside-the-machines/chapter-10-raising-a-helpful-assistant/">Read the full post →</a></p>https://www.fanfani.net/tutorials/inside-the-machines/chapter-11-how-models-run/Mon, 01 Jan 0001 00:00:00 +0000https://www.fanfani.net/tutorials/inside-the-machines/chapter-11-how-models-run/<p>Chapter 11 — How a Model Thinks in Real Time We have built a model (Chapter 9) and raised it into something helpful (Chapter 10). Now we watch it speak. This final chapter of the lifecycle is about inference — the moment-to-moment business of producing an answer from a frozen model. It is tempting to assume this is the simple part, the mere playing-back of what was learned. It is not. The act of generating is full of consequential choices that change the model&rsquo;s whole character, and it carries real costs that shape how these systems are built and served. Understanding inference is what separates a vague sense of &ldquo;the AI answers&rdquo; from a real grasp of what is happening.</p><p><a href="https://www.fanfani.net/tutorials/inside-the-machines/chapter-11-how-models-run/">Read the full post →</a></p>https://www.fanfani.net/tutorials/inside-the-machines/chapter-12-the-whole-landscape/Mon, 01 Jan 0001 00:00:00 +0000https://www.fanfani.net/tutorials/inside-the-machines/chapter-12-the-whole-landscape/<p>Chapter 12 — The Whole Landscape We have come a long way from &ldquo;a model is just a function.&rdquo; We have opened the box on five different kinds of system and walked the full arc from a pile of random knobs to a deployed, helpful assistant. This closing chapter does something the individual chapters could not: it steps back far enough to see the whole landscape at once — how the pieces relate, what they share, where they differ, and how to think about which one suits a given job. If you read nothing else after the foundations, read this.</p><p><a href="https://www.fanfani.net/tutorials/inside-the-machines/chapter-12-the-whole-landscape/">Read the full post →</a></p>https://www.fanfani.net/tutorials/inside-the-machines/appendix-a-field-guide-to-models/Mon, 01 Jan 0001 00:00:00 +0000https://www.fanfani.net/tutorials/inside-the-machines/appendix-a-field-guide-to-models/<p>Appendix A — A Field Guide to Real Models The main chapters described the kinds of model in the abstract. This appendix puts faces to the families: the real, named systems you will actually hear about, and the one specific thing each one contributed. The most useful way to read a family tree like this is not as a list of products but as a story of fixes — each model usually exists because someone looked at the previous one, named a single flaw, and corrected it. Read this way, the intimidating roll-call of names becomes a short, sensible history.</p><p><a href="https://www.fanfani.net/tutorials/inside-the-machines/appendix-a-field-guide-to-models/">Read the full post →</a></p>https://www.fanfani.net/tutorials/inside-the-machines/appendix-b-comparison-cheat-sheets/Mon, 01 Jan 0001 00:00:00 +0000https://www.fanfani.net/tutorials/inside-the-machines/appendix-b-comparison-cheat-sheets/<p>Appendix B — Cheat Sheets: How the Architectures Compare This appendix is a reference. Where the chapters told the story of each architecture, here we lay two of the most important contrasts side by side, in plain comparisons you can return to when you need to decide which tool fits a job. Two comparisons matter most: the reader against the writer, and the dense model against the mixture-of-experts. Reader versus writer The one idea everything flows from Every difference between a reader (an encoder, Chapter 5 ) and a writer (a decoder, Chapter 4 ) traces back to a single design choice: what is the model allowed to look at? A writer may only look backward, at what it has already produced or been given, never at what comes next. A reader may look in both directions at once, at the whole passage. That one rule cascades into everything else. Because the writer can only see backward, it can be trained to guess the next word and then actually produce text. Because the reader can see everything, it can build deep understanding — but it loses the ability to guess the next word, since the answer is sitting right there in front of it.</p><p><a href="https://www.fanfani.net/tutorials/inside-the-machines/appendix-b-comparison-cheat-sheets/">Read the full post →</a></p>https://www.fanfani.net/tutorials/inside-the-machines/appendix-c-attention-variants/Mon, 01 Jan 0001 00:00:00 +0000https://www.fanfani.net/tutorials/inside-the-machines/appendix-c-attention-variants/<p>Appendix C — A Field Guide to Attention Variants Chapter 3 introduced attention as the meeting where every word glances around at the others and decides which ones matter. That one mechanism is the whole game — but in real models it comes in several outfits, and the names can pile up into intimidating alphabet soup. This appendix is the decoder ring. The trick is to see that every &ldquo;attention variant&rdquo; is just that same meeting run under different rules, and that the rules answer practical questions: what does the task require, and what can the hardware afford? It helps to picture three independent dials — who is allowed to look at whom, how much memory the looking costs while writing, and how to keep the looking affordable when the text gets very long. Let us turn each in turn.</p><p><a href="https://www.fanfani.net/tutorials/inside-the-machines/appendix-c-attention-variants/">Read the full post →</a></p>https://www.fanfani.net/tutorials/inside-the-machines/appendix-d-positional-encodings/Mon, 01 Jan 0001 00:00:00 +0000https://www.fanfani.net/tutorials/inside-the-machines/appendix-d-positional-encodings/<p>Appendix D — Telling a Model Where Words Sit Chapter 2 mentioned, almost in passing, that a model has to be told the order of its words. This appendix unpacks that, because the question of how to inform a model about word order turns out to have a small, elegant history of its own — and it explains one of the quiet reasons modern models can handle ever-longer texts. Why order has to be spelled out at all Start with a quirk that surprises people: the attention mechanism, left to itself, is blind to word order. It sees a sentence as a bag of words with no sense of sequence — so &ldquo;dog bites man&rdquo; and &ldquo;man bites dog&rdquo; look identical to it, the same words tossed in a bag. But those sentences mean opposite things, and the difference lives entirely in the order. Since order carries real meaning, and the core machinery ignores it, someone has to hand the model the seating chart explicitly. Every scheme below is just a different way of telling each word where it sits. They run from clumsy to elegant, and each was invented to fix a flaw in the one before.</p><p><a href="https://www.fanfani.net/tutorials/inside-the-machines/appendix-d-positional-encodings/">Read the full post →</a></p>https://www.fanfani.net/tutorials/inside-the-machines/appendix-e-chinchilla-scaling/Mon, 01 Jan 0001 00:00:00 +0000https://www.fanfani.net/tutorials/inside-the-machines/appendix-e-chinchilla-scaling/<p>Appendix E — Compute-Optimal Scaling: The Chinchilla Story Chapter 9 touched on the question of how to split a training budget between a bigger model and more data, and named the Chinchilla correction. This appendix tells that story in full, because it is one of the most illuminating episodes in the field — a case where the conventional wisdom was confidently wrong, got corrected by a single careful study, and then turned out to have a twist that the correction itself missed.</p><p><a href="https://www.fanfani.net/tutorials/inside-the-machines/appendix-e-chinchilla-scaling/">Read the full post →</a></p>https://www.fanfani.net/tutorials/inside-the-machines/glossary/Mon, 01 Jan 0001 00:00:00 +0000https://www.fanfani.net/tutorials/inside-the-machines/glossary/<p>Glossary Every term and acronym used in this book, in plain language. Where a term is the everyday name for an idea explained in a chapter, the chapter is noted. Activated parameters — In a mixture-of-experts model, the small slice of the model actually switched on for a single word (the router plus the few chosen experts). Determines speed and running cost. Contrast with total parameters. (Chapter 7) Alignment (of a model) — The work of making a model behave the way people want: helpful, honest, well-mannered. Done through the shaping in Chapter 10. (Not to be confused with the translation sense below.)</p><p><a href="https://www.fanfani.net/tutorials/inside-the-machines/glossary/">Read the full post →</a></p>