Ask a model for the page number of a quotation it has never seen and it will give you one. Not a hedge, not a refusal, not a note that the request is beyond it — a page number, formatted correctly, inside a sentence of ordinary competent prose. The common reading is that the model is guessing badly, or was trained carelessly, or is in some loose sense pretending. None of those is quite right. The behaviour is what the training procedure asks for, at every position, and a model that did otherwise would have scored worse for it. That turns hallucination from a mysterious defect into a consequence of what the objective measures and what the available text contains. This piece is about why abstention is never the rewarded action, what the model’s own probabilities do and do not tell you about its uncertainty, and which of the available interventions actually change the situation.
The objective never rewards a refusal
Start with what the loss measures, because most accounts describe it loosely and the looseness hides the mechanism. During pretraining the model produces, at every position in a stretch of text, a distribution over the next token; the cross-entropy loss there is the negative logarithm of the probability it assigned to the token that actually came next. Every position is graded, on exactly one thing: the mass placed on the observed continuation.
Now go looking for the abstention. The vocabulary holds the pieces of “I do not know”, so the model can certainly emit that string. What it cannot do is be rewarded for emitting it where the text went on to say something else, because there the target is whatever the writer wrote, and mass spent on a hedge is mass taken from the target. Abstention is not merely unrewarded. It is penalized, whenever the text in front of it went on to make a claim.
The consequence is sharper than it first looks. Take a model completing a sentence that requires a citation it does not hold. If it emits a plausible-looking volume and page, the loss is the loss for getting those tokens wrong — precisely what a differently invented, equally plausible pair would have received. Nothing separates the two, and nothing charges either for the act of inventing. Fluent continuation is the whole of what is optimized. Accuracy is something it correlates with, on material the model absorbed, and correlation is all it ever is.
Nobody writes down what they declined to say
There is a second version of the argument, stronger than the first, and it concerns the data rather than the loss.
At training time the model is graded against text written by someone who knew the answer. That sounds obvious; its implications are not. A reference article states a population figure because its author had it to hand; a forum post about a compiler error was written by someone who had already found the cause. Where a writer did not know, the usual outcome is not a published sentence saying so but a sentence never written, a paragraph cut, a draft left in a drawer. The corpus is what survived a filter that removed most of the ignorance before publication.
So the behaviour a user wants — confidently declining a question the model nearly knows — has almost no support in the data. The model did not see that behaviour and learn to avoid it. There was very little of it there to see.
Hedging exists in text, of course, and this is where the point turns uncomfortable rather than merely surprising. Academic prose hedges constantly, forum answers say I think. Those hedges are properties of genre, not reports of a writer’s uncertainty about the fact at hand. A model that learns them acquires a register it can adopt on stylistic cues: when hedging sounds appropriate, which is a different function from whether this claim can be supported.
Figure 1. Why no path through training rewards a hedge.
Calibration is a real signal about the wrong object
The model is not wholly without information about its own uncertainty. The kind it has matters.
Calibration asks whether stated probabilities match observed frequencies: across the cases where a predictor said seventy percent, about seventy percent should have happened. Base models — the raw products of pretraining, before instruction tuning — are often reasonably calibrated over next tokens, which is unsurprising given a loss that rewards honest reporting. The information about uncertainty exists somewhere in the output distribution rather than nowhere at all.
Then the costs. A single token’s probability is not the probability that a claim is true. The quantity a user needs is over a proposition — that this person held that office in that year — and what the model produces is over tokens given a prefix. Different objects. A low probability may reflect only a choice among synonyms, with no doubt about the fact. A high probability may sit on a token that is merely the obligatory continuation of a sentence already committed to a falsehood. Getting from tokens to propositions means summing over every sequence expressing the claim, and generation does no summing.
Post-training tends to make the signal worse. Preference data comes from people comparing answers, and people reliably prefer answers that sound confident. A preference-optimized model is pushed toward the surface features hedging lacks, and the optimization obliges: instruction-tuned models are routinely more overconfident than the base models beneath them. The stage did what it was asked and degraded a property nobody measured.
More capable models relocate the problem rather than removing it. A stronger model hallucinates less on material it absorbed well, and more convincingly on material it did not: the invented citation acquires a real journal and an author who works in the field. So the error rate improves while the detectability of the errors that remain gets worse, moving in opposite directions on a quantity a reader depends on. An uncomfortable trade, and no phrasing of it reads well.
Three failures reported as one number
The word hallucination covers at least three mechanisms that share a symptom and little else.
There are failures of retrieval, where the material is in the weights and the wrong thing came out — the right sort of answer attached to the wrong entity, two biographies merged, a date shifted by a year. Memorization is partial and lossy, and these resemble the failures of a person recalling something read once.
There are failures of absence, where the material was never in the weights. Nothing was recalled because there was nothing to recall, and what appears instead is the shape such an answer takes, filled with tokens that fit it.
And there are failures of grounding, where a source was supplied and the output does not follow from it. The document said one thing, the answer says another, and the answer may even be true while unsupported by the material it claims to summarize.
These have different remedies, which is why separating them matters, and they are routinely reported as one hallucination rate. A number that improves may reflect better recall, broader coverage, or more faithful reading, and the evaluation rarely says which.
Retrieval changes the category rather than closing it. Putting relevant documents in the prompt, as retrieval-augmented generation does, converts absence into grounding. That is progress, since grounding failures can be checked against a source and absence failures cannot. It is not a solution, and the successor failure is specific: confident misreading of a supplied document, harder to notice than a fabrication because the citation resolves.
Sampling several answers detects instability, not error. Generate an answer several times at nonzero temperature, which the decoding settings make trivial, and see whether the versions agree. Disagreement is informative; a model offering four different dates is saying something useful. Agreement is not the complement of it. A model wrong the same way every time, which is what a systematic gap in the data produces, looks exactly like a model that is right.
Training an abstention installs the decline and its cost together. A model can be taught to say it does not know, through preference methods on data where declining is favoured, or a verifier rewarding refusal on unanswerable items. This works. What it buys is a decline that also fires on things the model did know, because the trained boundary is not the true boundary of its knowledge, and no placement avoids both errors at once. It is the two-sided trade any refusal carries.
Figure 3. Three failures, three remedies, one reported number.
A pinch of math: why abstention is never the optimal move
Two small calculations, pointing the same way.
The first acquits the objective. Write $q$ for the true conditional distribution over the next token in some context, and $p$ for the model’s. The expected loss incurred at that position is
$$\mathbb{E}_q[-\log p] = -\sum_{i} q_i \log p_i$$and this is a proper scoring rule: over all choices of $p$, it is minimized exactly when $p = q$. A model that shades its probabilities toward what sounds good does worse than one reporting what it estimates. The loss does not ask for overconfidence.
The trouble lies in what $q$ is defined over. That sum runs across the vocabulary, which contains no element meaning this question has no answer available to me. Whatever $q$ is, it is a distribution over continuations of written text, and in a corpus produced by people who knew things, the continuation at a position requiring a fact is the fact. There is nothing to put mass on that expresses ignorance.
The second calculation says where the damage lands. Suppose a claim occupies $k$ tokens, each drawn correctly with probability $p$, independently. The chance the whole claim is right is
$$p^k$$With $p = 0.95$ and $k = 20$, that is about $0.36$. The independence assumption is generous — tokens are correlated, and a model can recover from a bad one — but the exponent is the point. A one-word answer survives a moderate per-token error rate; a full citation or a case number does not. This is why fabrication clusters on names, numbers, dates and references rather than on the surrounding argument — on exactly the details a reader is most likely to check.
Figure 2. Why fabrication clusters on names and numbers.
What to carry forward
The training loss grades a model at every position on the probability it gave the token that actually came next, and no token in the vocabulary meaning I do not know would ever be the correct target in ordinary text, so abstention is penalized wherever the text went on to make a claim; a fabricated citation is charged exactly what a differently fabricated one would be, and neither is charged for the fabricating. The sharper form concerns the data: the corpus was written by people who knew the answers, since sentences composed out of ignorance rarely got published, so confidently declining a question one nearly knows is a behaviour with almost nothing to imitate — and the hedging that does appear belongs to genre rather than reporting on whether this claim can be supported. Uncertainty information does exist in the output distribution, and base models are often reasonably calibrated over next tokens, but a token’s probability is not the probability a proposition is true, and preference optimization erodes the calibration there was, since raters favour confident-sounding answers. Capability dissolves none of it: stronger models err less on well-covered material and err more convincingly elsewhere, so rate and detectability move in opposite directions. Failures of retrieval, of absence, and of grounding carry distinct remedies and one shared reported number. Retrieval converts absence into grounding, resampling detects instability but not systematic error, and a trained refusal declines some things the model did know. This is a property of the objective and of the available text rather than a defect of any particular model, so it is mitigated and not removed, and every mitigation is paid for in helpfulness.
The figures in this article follow the shared figure key , which explains what every shape, line style and colour means.
