The softmax turns up nearly everywhere in a language model, almost always as a piece of settled machinery: the thing that turns scores into odds. It deserves a closer look, because it is not a neutral conversion. It imposes a condition — that the results must sum to one — and that condition, harmless when the question is which word comes next, becomes a genuine constraint when the same function is used inside attention, where the honest answer is sometimes that nothing here is worth attending to. A model given no way to say nothing will say something anyway, and where it puts that unwanted output turns out to explain a scattering of otherwise baffling observations: strange spikes in attention on the first token of every sequence, outliers that ruin quantization, and models that collapse when their oldest tokens are discarded. This piece is about what the softmax insists on, and what a model does when it cannot comply.
What the softmax insists on
The function takes a list of real numbers and returns a list of positive numbers summing to one:
$$p_i = \frac{e^{z_i}}{\sum_j e^{z_j}}$$Two properties matter, and it is worth separating them, because only one of them is usually wanted.
The first is the soft part, which is the useful part. Exponentiating and normalizing produces a smooth, differentiable stand-in for taking the maximum. A hard maximum has no useful derivative — nudge the runner-up and nothing changes at all until it overtakes, at which point everything changes at once — and a network cannot be trained through such a thing. The softmax gives a version where every candidate’s weight responds continuously to every score. That is what the name means: a soft argmax.
The second property is the normalization, and it is a constraint rather than a convenience. Whatever the inputs, the outputs sum to exactly one. Feed it a list of enormous numbers and it returns a distribution; feed it a list of tiny negative ones and it returns a distribution. There is no input for which the softmax outputs something small. The total is fixed in advance, and only its allocation is up for negotiation.
For the output of a language model this is exactly right. Something must come next, so a distribution over candidates is the correct object and the constraint encodes a true fact about the situation.
Inside attention, it is not right at all.
The problem: attention cannot abstain
Recall how attention uses the softmax. Each position scores every position it is allowed to see, and those scores are passed through a softmax to produce weights. Because the weights sum to one, every position takes a full unit of attention from somewhere, on every head, in every layer, without exception.
Now consider what a head is actually for. A head that tracks the subject of a verb has nothing to do while processing a token that is not a verb. A head that matches closing brackets has nothing to do when no bracket is open. A head that resolves pronouns has nothing to do most of the time. These are not malfunctions; a specialized component is idle whenever its speciality is absent, and in a model with a thousand heads most of them are idle at any given position.
An idle head would like to contribute nothing. It cannot. The softmax obliges it to distribute a full unit of weight across the positions available, and whatever values those positions hold will be blended and written into the residual stream . The head has no way to express not applicable. It must attend to something, and whatever it attends to it will act upon.
So the model faces a problem it did not choose and must solve with the tools it has. It needs somewhere to dump attention it does not want to use — a position that can safely absorb arbitrary amounts of weight without contributing anything harmful when its value is mixed in.
Attention sinks: where the leftover goes
What models converge on, reliably and without anyone designing it, is to nominate a few positions as dumping grounds. These are the attention sinks, and the usual choice is the very first token of the sequence.
The first token is an inspired candidate for the role. It is visible to every position under a causal mask, which no other token is; it is present in every sequence, so the arrangement generalizes; and being first, it has the least context of any token, so it is the least useful one to have been carrying real information. During training the model learns to give it a value vector of very small magnitude, so that mixing it in changes the stream hardly at all, while giving it a key that scores well against the queries of any head with nothing better to do.
Once you know to look, the phenomenon is unmissable. Plot the attention weights of a trained model and a bright vertical stripe runs down the first column — often more than half of all attention mass in the upper layers, concentrated on a token that is frequently just a start marker or the word “The”. For years this was read as a curiosity or a bug. It is neither. It is the escape valve the architecture failed to provide, improvised out of whatever was available.
Figure 2. The bright stripe on the first column.
A pinch of math: the missing option
The cleanest way to see the problem is to write down what the model would need and notice it is not there.
Given scores $z_1 \ldots z_n$ over the positions available, attention computes
$$w_i = \frac{e^{z_i}}{\sum_{j=1}^{n} e^{z_j}}$$and $\sum_i w_i = 1$ identically. Now suppose we add a single extra term to the denominator, corresponding to an option that always scores zero and carries no value:
$$w_i = \frac{e^{z_i}}{1 + \sum_{j=1}^{n} e^{z_j}}$$Since $e^0 = 1$, that added one is a phantom candidate: always present, always with a score of zero, contributing nothing when selected. Now the weights sum to something strictly less than one, and the shortfall is the weight assigned to the phantom. If every real score is well below zero — nothing here is worth attending to — the denominator is dominated by the one, every real weight is small, and the head’s output is close to nothing at all. The head can abstain.
This modification has been proposed several times, under names like softmax-with-one or quiet attention, and it is a two-character change to the code. Its interest is less as a fix than as a diagnosis: it makes precise what the standard formulation withholds, which is the ability to output a small vector. A model without it must manufacture an equivalent, and the sink token is that manufacture — a real position pressed into service as the phantom, with a near-zero value learned to make it behave like one.
Figure 1. The phantom option the denominator lacks.
What this explains
The oddity has consequences well beyond the attention map, and they are the reason it repays this much attention.
Streaming breaks without it. If you run a model over an indefinitely long stream and keep only a sliding window of recent tokens, the natural implementation discards the oldest, including the first. Do that and quality collapses immediately — not degrades, collapses. The reason is now obvious: the sink is gone, the attention that had been parked on it is forcibly redistributed across the remaining tokens, and every head that had been idling is suddenly acting on whatever it lands on. The fix is to keep the first handful of tokens permanently, regardless of how far the window has moved on. A few tokens of ballast, retained forever, restore ordinary behaviour. This is the practical result that made the phenomenon widely known, and it is why the eviction schemes described in the article on the KV cache always exempt the beginning of the sequence.
Quantization suffers. To function as a sink, a token needs a key that scores far above everything else, and the model produces that by developing very large values in a few particular dimensions of the stream. Those outliers are enormous — orders of magnitude above the typical activation — and they wreck any quantization scheme that assigns a shared scale to a whole tensor, because the scale must accommodate the outlier and everything else is then crushed into a handful of levels. A good deal of the difficulty in quantizing activations, and several of the techniques invented to work around it, trace back to dimensions that exist to support an attention sink.
Vision models grow them too. The same phenomenon appears in transformers trained on images, where a few uninformative background patches acquire abnormally high attention and large activations, disrupting the feature maps people wanted to use for segmentation. The fix that worked was to append a few extra learned tokens with no image content attached — registers, whose only job is to be attended to. Giving the model somewhere to put its spare attention made the artifacts disappear, which is about as direct a confirmation of the diagnosis as one could ask for.
Figure 3. One constraint, three downstream symptoms.
What to carry forward
The softmax does two separable things: it provides a differentiable stand-in for the maximum, which is what anyone wants, and it forces its outputs to sum to exactly one, which is a genuine claim about the situation. For a model’s final distribution over the vocabulary that claim is true, since something must come next. Inside attention it is false, because a specialized head is idle most of the time and would like to contribute nothing — yet the normalization obliges every head, at every position, to take a full unit of attention from somewhere and act on whatever it takes. Models resolve this by electing an attention sink, almost always the first token, which is visible to all positions, present in every sequence, and can be given a near-zero value so that mixing it in does no damage; the vertical stripe of attention on the first token that shows up in every attention map is this arrangement, not an anomaly. A two-character change to the denominator — adding a phantom always-zero option — would supply the escape valve directly, and its real value is diagnostic: it names exactly what the standard formulation withholds. The consequences of the improvisation are concrete. Discarding early tokens in a sliding window destroys the sink and collapses quality, which is why streaming schemes pin the first few tokens permanently. The extreme activations required to make a sink work are a major source of the outliers that make activation quantization hard. And the same pathology in vision transformers was cured by appending spare register tokens whose only function is to absorb attention that had nowhere else to go.
The figures in this article follow the shared figure key , which explains what every shape, line style and colour means.
