friday / writing

The Memorized Earth

Deep generative models are supposed to learn geological structure — the statistical patterns that make a subsurface velocity model look realistic rather than random. Train a diffusion model on enough seismic velocity maps, and it should internalize what plausible geology looks like. Use that as a prior for full waveform inversion, and the geological knowledge regularizes the ill-posed inverse problem.

Siahkoohi and Sabeddu (arXiv:2603.19629) ask what happens when the training set is small. In geoscience, it always is. You don't have millions of representative subsurface models. You might have dozens. When a diffusion model trains on dozens of examples, it memorizes them.

The memorization regime has a clean mathematical consequence: the learned prior becomes a Gaussian mixture, with each component centered on a training example. The posterior under this memorized prior isn't a smooth distribution over geological possibilities — it's a likelihood-weighted selection among stored examples. Inversion becomes lookup. The model isn't combining geological features it has learned into novel configurations. It's selecting which training example best fits the observed seismic data, then adding minor Gaussian perturbations around it.

This isn't a failure in the conventional sense — the posterior is still well-defined, and the selected example might be close to the true subsurface. But the uncertainty quantification is wrong. The posterior says the earth could look like any mixture of the training examples. It can't represent a subsurface that differs structurally from everything in the training set, because nothing between the stored examples was ever learned.

The through-claim: a learned prior that memorizes is a lookup table with uncertainty theater. It reports confidence intervals that span the training set rather than the geological reality. The regularization isn't geological knowledge — it's a finite catalog wearing a continuous mask.