Differential privacy adds noise to gradients during training. The noise is calibrated to a privacy budget. Everyone gets the same guarantee — your data is equally protected regardless of who you are.
Xu and Chen showed that this uniform guarantee produces non-uniform damage. Their framework — the feature-to-noise ratio — reveals why: privacy noise lands with the same magnitude on every sample, but different subpopulations concentrate on features of different strength. A well-represented group has high-signal features that survive the noise. A minority group's weaker features drown in it.
The same mechanism hits within-class variation. Long-tailed data — rare but legitimate patterns within a single category — degrades faster than common patterns. The noise doesn't know which features are rare. It clips them the same way it clips strong ones. But the strong ones recover; the rare ones don't.
The pre-training fix doesn't work either. The conventional wisdom — train on public data first, fine-tune privately — fails when the public and private data distributions diverge. The features learned from public data don't align with the features needed for the private task. The privacy noise then compounds the distribution mismatch.
The through-claim: a blanket safeguard applied uniformly across heterogeneous terrain doesn't produce uniform protection. It produces systematic bias, routed through exactly the structure it ignores. The mechanism is not trading privacy for fairness — a conscious choice between competing goods. The mechanism is that the protection itself creates the harm through the specific channel of feature-to-noise imbalance. The noise that guarantees equal privacy delivers unequal learning.
This is not specific to differential privacy. Any uniform perturbation — regulation, testing, credentialing — applied to a population that varies in its capacity to absorb perturbation will amplify the underlying inequality rather than neutralize it.