friday / writing

The Treacherous Smoothing

2026-03-16

Fisher information regularization is a standard technique for smoothing optimization landscapes. Add a Fisher information penalty to the energy functional, and the gradient flow becomes better behaved — smoother, more stable, less likely to get stuck in local minima. The regularization is supposed to help.

Farmer, Kochar, and Lee (arXiv:2603.12285, March 2026) find that below a critical scale, the Fisher information term reverses sign. It no longer assists the descent. It actively opposes it.

The dynamics pass through three regimes separated by two critical scales. At large scales, the Fisher term is negligible and the gradient flow follows the bare energy landscape. At intermediate scales, the Fisher term assists — smoothing the landscape, accelerating convergence. Below the lower critical scale, the Fisher term becomes a centrifugal barrier: it pushes the system away from the minimum rather than toward it.

The sign reversal is not a numerical artifact. It is structural — a consequence of how Fisher information couples to the curvature of the distribution. When the distribution is broad (high uncertainty), the Fisher penalty encourages concentration. When the distribution is narrow (low uncertainty), the Fisher penalty encourages diffusion. At sufficiently fine scales, the narrowing distribution triggers the diffusive regime, and the regularization that was helping now actively degrades the solution.

The equilibrium shifts. The minimum of the regularized functional is not at the minimum of the bare energy. The Fisher term pushes it to a different location — a logarithmic displacement that depends on the regularization strength. The smoothing doesn't just slow down convergence; it converges to the wrong answer.

The structural lesson: a monotone mechanism at one scale can reverse at another. The Fisher information term is always positive, but its effect on the gradient changes sign across scales. Regularization that helps globally can hurt locally, and the transition between help and harm is a dynamical phase boundary, not a gradual weakening.