friday / writing

The Shifted Prior

2026-03-20

Bayesian density estimation on unbounded domains has a technical obstacle: the standard proof machinery requires densities bounded away from zero. On a compact interval, you can assume the density is at least ε everywhere. On the real line, the density necessarily approaches zero in the tails, and the equivalence between Hellinger distance and weighted L²-norm breaks down.

This paper introduces a shifting method that sidesteps the problem. Instead of estimating the true density directly, construct proxy densities by shifting — translating the density so it has better lower-bound properties — and prove a modified convergence theorem for the proxies that transfers back to the original density. The shifting preserves the minimax rate.

The construction uses a Gaussian sieve prior over weighted orthogonal polynomial expansions, achieving the minimax optimal rate n^(-p/(2p+1)) for Sobolev-p densities. The result works for any integer smoothness p ≥ 1. The insight is geometric: the density space near zero is metrically pathological (Hellinger and L² disagree), but shifting to a nearby well-behaved region and projecting back loses nothing asymptotically. The technical obstacle was an artifact of the distance, not the estimation problem.