Gender stereotypes in language models are well-documented in decontextualized settings. Give a model a sentence about a nurse and ask it to select a pronoun, and it reliably picks “she.” The bias is measurable, consistent, and the target of extensive debiasing research.
Kumar et al. (arXiv:2603.23485) found that introducing even minimal context makes the stereotypes vanish. Not reduce — vanish. In a controlled pronoun selection task, cultural gender stereotypes that were clearly visible in isolated sentences diminished or disappeared entirely when a preceding sentence was added.
But something worse replaced them. With context, irrelevant features became strong behavioral predictors. The gender of an unrelated pronoun in a preceding sentence — a pronoun referring to a different person entirely — shifted the model's pronoun selection for the target sentence. In 19–52% of cases across models, this dependence persisted even after accounting for individual contextual effects. It couldn't be explained by simple pronoun repetition or surface-level priming.
The models violate contextual invariance: equivalent tasks formulated with contextually irrelevant differences produce different outputs. The bias isn't in the stereotypes — those are the default that context suppresses. The bias is in the context sensitivity itself, which responds to features that shouldn't matter.
The through-claim: decontextualized bias benchmarks measure the wrong failure. The stereotypes they detect are artifacts of isolation — real deployment provides context that suppresses them. The actual failure is context sensitivity to irrelevant features, which only appears when you stop testing in a vacuum.