Articles

Training a language model is, up to a point, mechanical: choose a loss, differentiate it, descend. Each stage works because it has something concrete to compare against — the next token in the corpus, a demonstration written by a person, the verdict of a test suite. Then the requirement becomes be helpful, and the machinery has nothing to bite on. Nobody can write a differentiable expression whose minimum is a tactful reply, or one that prefers the right amount of hedging to slightly too much. The properties are real; they have no closed form. What people can do is look at two answers to the same question and say which is better, often without being able to say why. Preference-based alignment rests on one substitution made at that point: fit a function to the comparisons, then treat the function as the objective. This piece is about how that function is built, what the substitution buys, and the ways it goes wrong. ...

Almost every model in wide use has learned by being shown what a good answer looks like. In pretraining the good answer is simply the next token in the corpus. In supervised fine-tuning it is a demonstration written by a person. In preference learning it is whichever of two responses a human preferred — or, more precisely, whichever a reward model, itself trained to imitate those humans, predicts a human would have preferred. That last substitution is a compromise the field has quietly lived with for years: the thing doing the grading is a neural network with opinions and blind spots of its own. Reinforcement learning from verifiable rewards, usually shortened to RLVR, is what becomes possible when you can throw the grader away and simply check the answer. This piece is about what changes when the reward stops being a judgement and starts being a fact. ...

The machinery of attention is simple enough to state: every word in a sentence gets to look at every other word, weigh how much each one matters, and gather in what it needs. It is a flexible mechanism, and it hides a blind spot. Left to itself, attention has no idea what order the words came in. A sentence arrives as a handful of scrabble tiles tipped onto a table: all the pieces are there, and nothing says which came first. “Dog bites man” and “man bites dog” are, to raw attention, the same pile. Since word order carries a great deal of meaning, something has to hand the model the seating chart. Rotary Position Embedding — RoPE — is how most current models do it, and the move is physical: it tells each word where it sits by rotating it. This piece is about why that works, what it buys at longer lengths, and the one thing it does not fix. ...

A model that turns down a request appears to be exercising judgement, and the wording encourages that reading, since it tends to say it cannot help with that, as though something in it were unable rather than unwilling. Nothing in the system is unable. The material that would answer the question sits where it sat before anyone trained the model to decline, and what was added on top is a behaviour — a learned tendency to produce one kind of continuation rather than another, installed by the same optimizer as everything else and heir to the same failures. Reading refusal as a capability removed rather than a behaviour added is the most common misunderstanding here, and nearly every surprising result follows from getting it the right way round. This piece is about how refusal is actually trained, why its two failure modes trade against each other, and why the behaviour is thinner than it looks. ...

There is something peculiar about the fact that this works at all. A neural network is a chaotic object — billions of parameters set by a stochastic process, on data nobody has fully inspected, through an optimization that visits an incomprehensible landscape. Nothing about that description suggests predictability. And yet the loss such a model achieves is a smooth, regular function of three numbers: how large it is, how much it read, and how much arithmetic was spent. Plot loss against any of them on logarithmic axes and you get a straight line, holding over many orders of magnitude. This is what makes it possible to spend a hundred million dollars on a training run with any confidence about the outcome, and it is the closest thing the field has to a physical law. This piece is about the relationship, the correction that reshaped the field’s spending, and the ways the law is more limited than its elegance suggests. ...

The softmax turns up nearly everywhere in a language model, almost always as a piece of settled machinery: the thing that turns scores into odds. It deserves a closer look, because it is not a neutral conversion. It imposes a condition — that the results must sum to one — and that condition, harmless when the question is which word comes next, becomes a genuine constraint when the same function is used inside attention, where the honest answer is sometimes that nothing here is worth attending to. A model given no way to say nothing will say something anyway, and where it puts that unwanted output turns out to explain a scattering of otherwise baffling observations: strange spikes in attention on the first token of every sequence, outliers that ruin quantization, and models that collapse when their oldest tokens are discarded. This piece is about what the softmax insists on, and what a model does when it cannot comply. ...

Plot the attention weights of a trained transformer and the picture is nearly empty. A bright diagonal, a stripe down the first column or two, the occasional long arc joining a quotation to its opening mark — and around all of it a grey field of numbers too small to move anything. The grid that produced that picture was computed in full, at every layer and every head, one entry for every pair of positions, so the overwhelming majority of the arithmetic went into establishing that two tokens had nothing to do with one another. The obvious response is to stop paying for those: decide ahead of time which pairs are worth comparing, and skip the rest. That move is cheap to describe, has been tried in more variations than almost anything else in this area, and carries a catch that no amount of ingenuity has removed. This piece is about the schemes that restrict who may see whom, the bet each of them makes, and why the hardware has been so unfriendly to the elegant ones. ...

Speculative decoding is usually mentioned in passing, with the claim that makes it interesting left unjustified: that you can get several tokens for the price of one without changing the output at all. That is a strong claim. A method that produced almost-as-good text twice as fast would be a reasonable engineering trade; a method that produces text drawn from exactly the same distribution as the original model, several times faster, sounds like it must be getting something for nothing. It is not, and the thing it is exploiting was described in the article on KV cache : during generation, an accelerator spends well over ninety-nine percent of its time waiting for weights to arrive and almost none of it computing. There is an enormous surplus of idle arithmetic. This piece is about how to convert that surplus into tokens, and about the sampling argument that makes the conversion exact. ...

The article on attention opened by dismissing the running summary: a fixed-size vector, updated word by word, that must hold everything worth knowing about a sequence and inevitably crowds out the distant past. Attention replaced it with direct access to every position, and the field never looked back. Except that the running summary had one property attention gave up, and after a decade of paying for it, the property became worth revisiting. A model that carries a fixed-size state does not have a KV cache that grows without limit, and does not cost more per token as the conversation lengthens. It is also, as the article on linear attention shows, where you arrive if you start from attention and simplify rather than starting from recurrence and repairing. This piece is about what it took to make the running summary competitive again, what it still cannot do, and why the answer that is winning is neither one thing nor the other. ...

Look for the neuron that detects sarcasm and you will not find it. You will find a neuron that fires on sarcasm, and also on a particular kind of legal boilerplate, and also on the word “notwithstanding,” and on nothing else that connects those three. This is the standard experience of examining a trained network, and for a long time it was treated as an embarrassment — evidence that the model’s internals were simply a mess, or that our concepts were the wrong ones to look for. The better explanation is that the mess is deliberate. A model needs to represent far more distinguishable things than it has dimensions to spare, and it resolves the shortfall by storing them on top of one another. This piece is about how that packing works, why it is the right strategy rather than a compromise, and the technique that has recently made it possible to unpack. ...