Look for the neuron that detects sarcasm and you will not find it. You will find a neuron that fires on sarcasm, and also on a particular kind of legal boilerplate, and also on the word “notwithstanding,” and on nothing else that connects those three. This is the standard experience of examining a trained network, and for a long time it was treated as an embarrassment — evidence that the model’s internals were simply a mess, or that our concepts were the wrong ones to look for. The better explanation is that the mess is deliberate. A model needs to represent far more distinguishable things than it has dimensions to spare, and it resolves the shortfall by storing them on top of one another. This piece is about how that packing works, why it is the right strategy rather than a compromise, and the technique that has recently made it possible to unpack.
More features than dimensions
Start with the counting problem. The residual stream is a few thousand numbers wide. The things a model might usefully represent about a piece of text number vastly more: the topic, the register, the language, the syntactic role of the current token, whether a quotation is open, which of a hundred thousand entities is being discussed, whether the tone is hostile, whether this is code and in which language, and so on without obvious limit. Any reasonable estimate of the useful features runs to millions.
If each feature needed its own dimension, the model could hold a few thousand. It evidently holds far more. So features cannot be occupying dimensions one apiece.
The naive alternative — that features are simply spread across all dimensions in some hopelessly entangled way — is also wrong, and it is worth knowing why, because the evidence is strong. Directions in the stream behave linearly: you can find a direction corresponding to a concept, add it to a model’s activations mid-forward-pass, and the model’s behaviour shifts in the corresponding way. Sentiment, formality, refusal , even the language of the output can be steered like this. That would not work if concepts were smeared arbitrarily. Features live along directions; there are just more directions in use than there are dimensions.
How you fit more directions than axes
In a space of $d$ dimensions you can have exactly $d$ mutually perpendicular directions. But if you relax the requirement from perpendicular to nearly perpendicular, the count explodes.
This is a consequence of high-dimensional geometry that has no low-dimensional intuition behind it. In two or three dimensions, vectors crowd each other immediately. In a thousand dimensions, you can find exponentially many vectors that are pairwise almost orthogonal — the number grows as roughly $e^{\Theta(\epsilon^2 d)}$ for a tolerance $\epsilon$ on the inner product. A thousand-dimensional space comfortably holds many thousands of directions with pairwise overlaps small enough to ignore.
So the model assigns each feature a direction, allows those directions to overlap slightly, and accepts a small amount of interference between every pair. This is superposition: representing $n$ features in $d < n$ dimensions by using near-orthogonal directions and tolerating the resulting noise.
The reason it works in practice, rather than merely in geometry, is sparsity. Most features are absent from any given input. A text is not simultaneously about French grammar, mortgage law, and Baroque music; the overwhelming majority of the model’s features are inactive at any moment. Interference only matters between features that are active at the same time, and if only a handful are active at once, the accumulated noise from their overlaps stays small. The sparser the features, the more of them can be superposed. This is the precise sense in which superposition is a good strategy rather than a compromise: given sparse features, it stores strictly more useful information per dimension than dedicating axes would.
Figure 1. Nearly perpendicular is enough.
A pinch of math: why the model chooses this
The trade can be written down, and doing so shows that superposition is what an optimizer should find.
Suppose feature $i$ has a direction $v_i$ of unit length, and is present with probability $p$ and magnitude $a$. A reader looking for feature $i$ projects onto $v_i$. When feature $i$ is present it gets the signal $a$. It also gets interference from every other active feature $j$, amounting to $a \sum_j (v_i \cdot v_j)$ over the active ones.
With $n$ features in $d$ dimensions and directions arranged as well as possible, the typical overlap between two of them is on the order of $1/\sqrt{d}$. The expected number simultaneously active is $pn$. So the interference accumulates to roughly
$$\text{noise} \;\sim\; a \sqrt{pn} \cdot \frac{1}{\sqrt{d}} \;=\; a\sqrt{\frac{pn}{d}}$$taking the square root because the overlaps have random signs and partially cancel. The signal-to-noise ratio is therefore about $\sqrt{d / pn}$, and the condition for the representation to be usable is
$$pn \;\lesssim\; d$$Read that as the governing inequality of the whole subject. What must fit within the dimension count is not the number of features but the number typically active at once. A model can hold a hundred thousand features in four thousand dimensions provided only a few dozen are ever live together. Dedicating one axis per feature would cap the total at $d$ and waste nearly all of them on features that are absent; superposition trades a little noise for an enormous increase in capacity, and when features are sparse the trade is overwhelmingly favourable.
This has been demonstrated directly in small models built for the purpose: train a network to reconstruct sparse inputs through a narrow bottleneck, and it will represent more features than the bottleneck has dimensions, arranging them in regular geometric configurations — digons, triangles, pentagons, tetrahedra, square antiprisms — that maximize the minimum angle between them, structures related to the classical problem of spacing points evenly on a sphere. As the inputs are made denser the model abandons superposition and reverts to representing only its most important features, one per dimension, discarding the rest entirely. The strategy is selected by the sparsity, exactly as the inequality predicts.
Figure 2. What must fit is what is active at once.
Why neurons look confused
The consequence for interpretation is immediate. If features occupy directions that are not axis-aligned, then a single neuron — a single coordinate of the stream — is a projection onto one axis, and that axis will have some overlap with many different feature directions. A neuron therefore fires for several unrelated features.
This is polysemanticity, and it is not a defect but the visible shadow of superposition. The neuron responding to sarcasm, legal boilerplate, and “notwithstanding” is not confused; it is one arbitrary coordinate in a space where three unrelated features happen to have components. Nothing forced the model to align its features with the coordinate system, and there was no reason for it to.
Which means that a great deal of early interpretability work was attempting something impossible. Cataloguing what each neuron does, on the assumption that neurons are the units of meaning, cannot succeed if the units of meaning are directions that cut across neurons. The unit was wrong.
Unpacking it
The recovery technique follows directly from the diagnosis. If the model has compressed many sparse features into few dimensions, then decompressing means finding an overcomplete set of directions — more directions than dimensions — such that any given activation is a combination of only a few of them.
That is a well-posed problem, and the tool is a sparse autoencoder: a network trained to take a model’s activation vector, expand it into a much wider layer, and reconstruct the original from that wide representation, under a penalty that forces the wide layer to be mostly zeros. Reconstruct accurately, but with few active units. The trained expansion’s units then correspond to individual directions in the original space, and because sparsity was enforced, they tend to correspond to the features the model was superposing.
They are strikingly more interpretable than neurons. Where a neuron fires on three unrelated things, a learned feature will fire on one recognizable thing — a specific person, a code construct, a rhetorical move, the state of being inside a quotation. Millions of such features have been extracted from production models, and they can be manipulated: amplify one and the model’s output shifts accordingly, which is the confirmation that the feature is causally involved rather than merely correlated.
The honest caveats matter. Reconstruction is imperfect, and the part that is lost may not be the unimportant part. The number of features is a choice, and different choices split or merge concepts differently, with no principled way to know the right granularity. There is evidence of features that are compositions of others rather than atoms. And scaling the technique to cover a whole model, at every layer, remains expensive. It is the best tool available for looking inside these systems and it is not a solved problem.
Figure 3. Compressed by the model, expanded by the autoencoder.
What to carry forward
A model needs to represent far more distinguishable things than it has dimensions, and it does so by giving each feature a direction rather than an axis, allowing those directions to be merely near-orthogonal rather than perpendicular. High-dimensional space permits exponentially many such directions, and the resulting interference stays tolerable because features are sparse — only a handful are active at any moment, and interference only matters between simultaneously active features. The governing condition is that the number of features typically active at once must fit within the dimension count, not the total number of features, which is why a few thousand dimensions can carry a hundred thousand concepts. This makes superposition the correct strategy rather than a compromise: dedicating one axis per feature would cap the total at the width and squander nearly all of it on features that are absent. The visible consequence is polysemanticity — a single neuron is one arbitrary coordinate that overlaps many feature directions, so it fires on unrelated things and is not the unit of meaning, which is why cataloguing neurons was attempting something impossible. The unpacking technique follows from the diagnosis: train a sparse autoencoder to reconstruct activations through a much wider layer constrained to be mostly zero, and its units recover the superposed directions as individually interpretable features that can be amplified to confirm they are causally involved. Reconstruction is lossy, the right granularity is a free choice with no principled answer, and covering an entire model remains expensive.
The figures in this article follow the shared figure key , which explains what every shape, line style and colour means.
