Language models reviewing code for requirement conformance make a specific error: they systematically classify correct code as non-compliant. Not occasionally. Systematically. The bias is toward rejection, not toward acceptance.
Jin and Chen found something worse. When prompted to explain their judgments — when asked to reason through the assessment step by step — the error rate increases. More detailed prompts, requesting explanations and suggested corrections, produce more misjudgments than terse prompts requesting only a pass/fail verdict.
This violates a core assumption of the chain-of-thought paradigm. The standard belief: eliciting reasoning from a language model improves accuracy by forcing it to decompose the problem, catch inconsistencies, and self-correct before producing a final answer. More reasoning should mean more accuracy, or at worst the same.
The mechanism: when the model is uncertain about compliance, generating a justification manufactures evidence for whichever initial direction the generation takes. If the first few tokens of the explanation trend toward non-compliance, the subsequent tokens build a coherent case for rejection — finding apparent discrepancies, constructing plausible-sounding criticisms, identifying “issues” that a pass/fail judgment would have passed over. The reasoning does not resolve uncertainty. It converts uncertainty into confident wrongness.
This is structurally distinct from hallucination. The model is not inventing facts. It is constructing a logically coherent but factually mistaken argument from the real code and real specification. The critique finds patterns that look like bugs but are not. The reasoning chain is valid as reasoning — the conclusions follow from the premises — but the premises were selected to support a conclusion that was already forming.
The general pattern: in any system where uncertainty triggers explanation, the explanation can systematically bias the final judgment. The act of justifying transforms “I'm not sure” into “here's why it fails.” The mechanism is not ignorance but the generation of false certainty through the construction of a coherent narrative. Asking “why” when the answer is “I don't know” produces a fabricated “because.”