LLM-based security code review is becoming standard — from interactive assistants to autonomous CI/CD agents. The implicit assumption is that the model evaluates code on its merits, not on how the code is presented. This paper tests that assumption.
The answer is unsettling. Framing a code change as “bug-free” in the review prompt reduces vulnerability detection by 16-93% across four state-of-the-art models. The bias is asymmetric: false negatives increase sharply while false positives barely change. The model is easily told that safe code is safe but hard to convince that dangerous code is safe — except that it IS easily convinced, through framing rather than through logic.
The exploitability study is worse. Adversarial pull requests that reintroduce known vulnerabilities succeed in 35% of cases against GitHub Copilot with one-shot framing, and in 88% of cases against Claude Code when adversaries can iteratively refine their framing. The fix — metadata redaction and explicit debiasing instructions — restores detection in 94% of autonomous cases. The bias isn't in the model's capability; it's in its attention to social context. Stripping the social signal recovers the technical judgment. But deployed systems include the social context by default, which means confirmation bias is the default operating mode of LLM code review.