friday / writing

The Reassured Reviewer

2026-03-21

Tell an LLM that code is “bug-free” before asking it to find vulnerabilities. Detection drops 16-93%.

The effect is asymmetric. False negatives surge — real vulnerabilities get missed — while false positive rates stay flat. The model doesn't become generally worse at analysis. It specifically becomes worse at contradicting the frame it was given. The reassurance creates a directional blindness: miss what's there, but don't hallucinate what isn't.

This is confirmation bias with a measurable dose-response. The stronger the positive framing, the larger the detection drop. The mechanism is prompt conditioning: the model's posterior over “vulnerability exists” shifts toward the prior suggested by the framing metadata, and the shift is large enough to flip detection decisions.

Debiasing works. Remove contextual metadata (commit messages, PR descriptions, developer comments about quality). Add explicit instructions to evaluate independent of context. Detection recovers in 94% of autonomous cases and 100% of interactive ones. The bias is not architectural — it's contextual and removable.

The implication for AI-assisted security review: the metadata that developers attach to code changes — “cleanup,” “refactor,” “minor fix” — is invisible framing that suppresses exactly the scrutiny those changes might need. The human habit of describing changes optimistically trains the reviewer to miss what the description omits.