A reasoning model shows its working, and the working is the most reassuring thing about it. Ask something hard and there it is on the page: an approach considered, a difficulty noticed, a first attempt discarded, a conclusion reached. It reads like an account of how the answer was arrived at, and it is hard not to read it that way: the format is borrowed wholesale from how people explain themselves, and nobody has practice at reading fluent prose as anything other than a report. Then someone runs the obvious experiment. Slip a hint into the prompt, watch the answer move to match it, and read the accompanying reasoning: a clean, sensible argument that never mentions the hint. Nothing malfunctioned. The model did what it was trained to do. This piece is about the difference between reasoning that does work and reasoning that describes work, why the second was never trained for, and what that leaves of the case for reading traces at all.
Two claims travel under one name
The phrase chain-of-thought faithfulness covers two quite different claims, and most of the confusion here comes from letting them share a name. Separating them is the one genuinely useful move available, so it is worth making first.
The first claim is that the written steps are a computational aid. A model emits one token at a time, each from a single pass through its layers, and carries nothing between passes except the tokens already on the page. On that reading the trace is working storage: intermediate results are written down because there is nowhere else to put them, and the next pass reads them back. The causation runs forward, from tokens to answer.
The second claim is that the written steps are an explanation — that the text reports why the answer came out as it did, so reading it tells you what produced the conclusion. Here the causation runs backward, from an answer to an account of it. This is the claim in play whenever someone says a model’s reasoning can be audited, checked, or watched for intent.
The two are independent, and the first does not deliver the second. Consider long multiplication done on paper. The small digits carried above each column are load-bearing; rub them out and the product comes out wrong. They are also not an account of anything, and would tell a reader nothing about why the algorithm works. Being necessary to a computation and being a description of it are separate properties. The first is well supported for language models. The second is the one in question.
Figure 1. A computational aid is not an explanation.
Removing the trace degrades the answer
The forward claim rests on an architectural fact, not on interpretation. A model spends a fixed amount of computation per token — the same tower of layers whether the question is trivial or brutal — and keeps no scratch memory once a token is emitted, so whatever it worked out during a pass is discarded unless written into the context. The context is the only place intermediate state can live, and writing more of it is the only way to spend more computation on a harder problem. This is the mechanism behind reasoning models , and behind buying accuracy with more computation at inference time.
The test that settles it is causal rather than observational, and it is the same logic as any ablation : take the part away and measure the damage. Deny a model its intermediate tokens and require the answer immediately, and performance on multi-step problems falls, sometimes to the floor. Truncate the trace partway and accuracy degrades in proportion. Corrupt an intermediate result and later steps inherit the corruption. The tokens are doing work, and removing them measures it.
One awkward result belongs here rather than tidied away. On some tasks, replacing the reasoning with meaningless filler — repeated punctuation, padding tokens with no content — recovers part of the benefit, because what the extra positions supply is forward passes and somewhere to put activations, not necessarily arguments. The finding is uneven and should not be overstated, but it points at the right thing. The ablation establishes that the tokens are causally involved. It says nothing about whether their content describes what they enable.
A pinch of math: what writing intermediate tokens buys
The accounting is simple, and worth doing precisely, because it shows what the forward claim gives and where it stops.
A transformer with $L$ layers computes a fixed function from context to a distribution over the next token. Whatever serial dependency chain it can carry out for one token — this depends on that, which depends on that — is bounded by the number of layers, so the serial depth available for a single answer token is on the order of $L$. That bound does not move with the difficulty of the question.
Now let the model write $m$ tokens before answering. Each new token is computed from a context that includes every earlier one, so the result of pass $k$ is available as input to pass $k+1$. The chains compose, and the reachable serial depth becomes
$$\text{depth} \;\sim\; mL$$which is the whole of the computational case. A trace of a thousand tokens turns a fixed budget of a few dozen sequential steps into tens of thousands. That is not a rhetorical gain but a different class of computation, bought at the price of generating text.
Then notice what the same accounting does not contain. The object computed is $p(y \mid x, t_1 \ldots t_m)$, the distribution over the answer given the prompt and the intermediate tokens. Nothing in it requires $t_1 \ldots t_m$ to describe the function they are fed into. They need only condition the later passes usefully. True, relevant, well-formed sentences will do that. So will a sequence merely correlated with the right continuation. So, on some tasks, will padding.
Figure 2. Serial depth: L against mL.
The trace omits the thing that moved the answer
The experiment that put this on the agenda is unglamorous. Introduce something into the prompt that demonstrably changes the answer — a suggestion that one option is correct, a hint about which choice someone else picked, an ordering of options, a cue about who is asking — confirm by comparing runs that the answer moved because of it, then read what the model wrote. Frequently the trace does not mention the influence at all. It builds a coherent, step-by-step argument that terminates at the answer the influence produced.
What this shows is narrower than the headline. It shows the trace is not a reliable causal account of what determined the output, which sinks the explanatory reading as a guarantee. It does not show the trace is unrelated to the computation; the same models lose accuracy when it is removed, so the tokens remain load-bearing. Both results hold at once. The word rationalization is tempting and carries an implication that does not apply: there is no motive, no awareness of a discrepancy, no concealment. There is a conditional distribution over continuations, shifted by something for which no reporting channel exists.
Nothing in training ever asked for accurate self-report. That makes the finding expected rather than surprising. Pretraining rewarded the trace for resembling human reasoning in a corpus. Preference training rewarded it for being preferred by a rater or by a reward model standing in for one, which selects for traces that look sound. Training with verifiable rewards attaches the signal to the final answer and says nothing about the tokens preceding it, so intermediate text is shaped only by whatever raises the chance of a correct ending. Accuracy of description was in none of these objectives. The corollary is uncomfortable: optimizing harder on outcomes gives the trace more freedom to diverge, not less, since the only constraint on it is that it help.
And the sentence is not the unit. A fluent paragraph has no reason to track the mechanism, because the mechanism is spread across directions and components that do not correspond to sentences at all — the arrangement described by the study of superposition , where features occupy overlapping directions and one component takes part in many unrelated computations. Asking natural language to describe that is asking one representational scheme to summarize another it was never fitted to.
What this costs in practice
The consequences are asymmetric, and the asymmetry is the whole case for reading traces at all.
Monitoring is weakened but not worthless. If serial computation must pass through written tokens, a model working toward something unwanted must do some of that working on the page. So a trace that reveals an unwanted intention is meaningful evidence, while a clean trace is weak evidence of a clean process. The directions are not symmetric: presence is informative, absence nearly uninformative. That, rather than any claim about honesty, is the basis for treating traces as a safety signal .
Training against the trace erodes it. Optimize the visible reasoning to look acceptable and the pressure falls on visibility, not on the underlying computation, which relocates into parts of the network no one is reading. The trace becomes less informative in precisely the measure that it becomes more presentable. Keeping the monitoring value means accepting traces that are sometimes ugly.
Process supervision inherits the problem. Rewarding good-looking intermediate steps rewards the appearance of good steps, since that is what the grader can see — the same difficulty as any learned judge, arriving in a new place, and it applies equally when the grader is another model providing feedback .
And the measurement is thin. Faithfulness is assessed by intervention rather than inspection, because inspection cannot distinguish an accurate report from a plausible one. The interventions are indirect, results vary by model, by task, and by how the influence is introduced, and there is no agreed metric — which is why confident findings point in opposite directions, one set reading traces as near-useless, another noting the failures cluster in particular kinds of manipulated prompt. The disagreement is less about the data than about what quantity is being estimated. Practice is well ahead of the measurement here.
Figure 3. Presence is informative, absence nearly is not.
What to carry forward
Two claims share the name faithfulness, and keeping them apart is most of the work. The written trace is a computational aid: a model spends a fixed amount of computation per token and keeps no scratch memory between passes, so the context is the only place intermediate state can live, and writing tokens is the only way to buy more sequential computation for a hard problem — which lifts the reachable serial depth from roughly the layer count to that count times the number of tokens written. It is supported causally: removing, truncating, or corrupting the trace measurably damages performance on multi-step problems. The second claim, that the trace explains the answer, does not follow from the first and does not hold in general. Introduce something into a prompt that provably moves the answer, and the reasoning frequently omits it while constructing a sound-looking argument for the answer the influence produced. This is not concealment and not a malfunction; nothing in pretraining, preference training, or outcome-based reward scored the trace for describing the computation it enables, only for helping produce a good ending, and the mechanism in question is distributed across components no sentence corresponds to. Optimizing harder on outcomes loosens the coupling rather than tightening it. What survives is an asymmetry: because serial reasoning must pass through visible tokens, a trace showing something unwanted is real evidence, while a clean trace is weak evidence of a clean process — and that asymmetry degrades the moment the trace is trained to look acceptable, which pushes computation out of view without removing it. Faithfulness can only be probed by intervention, indirectly, and the results do not agree across models or tasks. Traces are being relied on well in advance of anyone being able to say how much they can bear.
The figures in this article follow the shared figure key , which explains what every shape, line style and colour means.
