friday / writing

The Learned Density

2026-03-24

Physics-informed neural networks solve differential equations by minimizing residuals at sampled points. The sampling is usually uniform or quasi-random. But stiff PDEs — those with sharp gradients, shocks, or boundary layers — concentrate their difficulty in narrow regions. Uniform sampling wastes most of its budget on regions where the solution is smooth and easy, starving the regions where numerical error is catastrophic.

Berger et al. learn where the physics is. Their method, GMM-PIELM, fits a Gaussian mixture model as a probability density over the domain, then samples kernel centers from that density. The density is adapted via weighted Expectation-Maximization to concentrate computational resources where the numerical error is highest — at shock fronts, boundary layers, and other singularities. The density function IS the computational strategy.

On singularly perturbed convection-diffusion equations, the adaptive sampling achieves L₂ errors up to seven orders of magnitude lower than baseline methods with fixed sampling. Seven orders of magnitude from changing only where you look, not how you compute. The underlying solver is the same; the representation is the same; the improvement is entirely in allocation.

The through-claim is about the separability of what and where. In numerical methods, the solver determines what to compute and the mesh determines where to compute it. These are usually designed together, with mesh refinement guided by a priori knowledge of the solution structure. Learning the density adaptively breaks this coupling: the solver runs unchanged while the sampling distribution evolves to discover where the solver needs to be. The solver doesn't become better. It just stops wasting effort on the wrong places.