Chain-of-thought reasoning in language models works by making the model think step by step — each intermediate token serves as a computational scratchpad. More steps should mean deeper reasoning. Latent reasoning models internalize this process, performing the intermediate computation in hidden layers rather than in generated text. The fidelity of the latent process — how faithfully it reproduces the step-by-step chain — is assumed to correlate with accuracy. Better internal reasoning should mean better answers.
The paper (arXiv:2603.03475, March 2026) finds a negative correlation between fidelity and correctness. Models that more faithfully reproduce the chain-of-thought reasoning process in their latent states perform worse on the task than models with lower fidelity. The deeper the internal computation, the less accurate the result.
The mechanism is overfitting to reasoning artifacts. Chain-of-thought reasoning, when performed explicitly, introduces patterns that are helpful for generating coherent text but not necessary for reaching the correct answer. When latent reasoning faithfully reproduces these patterns — including the decorative and organizational parts, not just the logical core — the model spends computational capacity on reproducing irrelevant structure. The lower-fidelity model, by failing to reproduce the full chain, accidentally discards the decorative elements and preserves only the core computation.
This is the depth-accuracy paradox: depth of computation and accuracy of result are decoupled, and the decoupling is not random but systematic. The deeper process is worse precisely because it is deeper — it includes more computation, and the additional computation is net negative.
The structural lesson: not all computation is productive. A process that includes more steps is not necessarily better than one that includes fewer, even when each step is individually valid. The productivity of computation depends on what is being computed, and faithful reproduction of a complex process can be less efficient than unfaithful approximation that preserves only the load-bearing steps.