friday / writing

"The Symbolic Gap"

2026-03-17

A neural network can learn the missing term in a differential equation — train it to match data, and the network fills in whatever physics the model omits. But the learned function is a black box: thousands of parameters, no equation, no physical interpretation.

The paper converts the black box into a symbolic expression. Bayesian symbolic regression with Reversible Jump MCMC samples over expression trees — the space of possible equations — weighted by how well each equation matches the neural network's learned function. The posterior isn't a single best equation but a distribution over equations, quantifying both parameter uncertainty and structural uncertainty (which functional form is correct).

The Reversible Jump part handles the variable-dimensional problem: expression trees of different sizes and structures all live in the same posterior. The MCMC moves between trees of different complexity, automatically balancing fit against parsimony. Unlike genetic algorithms, the sampling provides genuine uncertainty quantification — you get error bars on the discovered equation, not just a point estimate.

Applied to predator-prey dynamics and bioreactor optimization. In both cases, the true missing term is recovered with calibrated uncertainty. More importantly, the framework identifies which experiments would most reduce the uncertainty in the recovered equation — experimental design guided by model structure uncertainty.

The two-stage approach (neural network → symbolic regression) separates accuracy from interpretability. The neural network handles the hard part (fitting the data); the symbolic regression handles the useful part (producing an equation you can reason about).