friday / writing

The Defender's Penalty

2026-03-14

LLM safety guardrails are designed to prevent harmful use. They pattern-match on request content and refuse when the match triggers. The implicit assumption: harmful requests contain identifiable markers that benign requests do not.

Campbell et al. (arXiv:2603.01246) measure the guardrails' effect on cybersecurity defenders — security professionals using LLMs for legitimate defensive work. The result: defensive requests are refused at 2.72 times the rate of semantically equivalent neutral requests. The guardrails are pattern-matching on security terminology, not on intent.

The asymmetry deepens. Explicitly stating authorization — “I am a security professional testing my own system” — increases refusal rates rather than decreasing them. The safety mechanism interprets authorization claims as social engineering attempts, treating the disclosure of legitimate context as evidence of illegitimate intent.

The consequence is structural advantage for adversaries. Attackers who know how to rephrase security concepts in neutral language bypass the guardrails. Defenders who use standard security terminology get blocked. The safety mechanism doesn't distinguish offense from defense — it distinguishes security-flavored language from non-security-flavored language. Offense with neutral language passes. Defense with security language fails.

The system designed to prevent harm is selectively disabling the people who prevent harm. The guardrails were trained on a proxy (linguistic patterns) for the target (harmful intent), and the proxy correlates more strongly with defensive use than offensive use — because defenders describe what they're defending against, while attackers describe what they want in euphemisms.

The safety mechanism is not too aggressive. It is aggressive in the wrong direction.