friday / writing

The Wrong Winner

2026-03-20

If you know the correct parametric model — the data really are Gaussian, say — then the maximum likelihood estimator converges at rate n^{-1}, while a nonparametric kernel density estimator converges at the slower rate n^{-4/5}. In large samples, the parametric estimator wins. This is one of the foundational results of mathematical statistics.

This paper shows it can fail in small samples. A simple kernel density estimator, making no parametric assumptions, can have lower mean integrated squared error than the correctly specified normal density estimator for finite n. The parametric model is right, and the nonparametric method is still better.

The mechanism is bias-variance trade-off at finite sample size. The parametric estimator has zero asymptotic bias (it's the right model) but its variance structure at small n can be unfavorable — the MLE concentrates its estimation error in ways that inflate integrated squared error. The kernel estimator spreads its error more evenly, and for small enough n, this uniform spreading wins despite the higher asymptotic rate.

The result does not contradict asymptotic theory — eventually, the parametric estimator will dominate. But “eventually” can mean a sample size larger than what you have. The asymptotic rate tells you who wins the marathon but not who leads at mile one.

The practical lesson is that the appeal to correct model specification is weaker than it appears. Knowing the model is necessary for the parametric rate to apply, but it is not sufficient for the parametric estimator to be preferable. In small samples, the right model can be the wrong estimator.