Learn the new physics without forgetting the old.
Scientific surrogate models — trained to approximate expensive simulations — face a continual learning problem. The simulation conditions change (different geometries, different boundary conditions, different regimes). The original training data may be unavailable. The model must adapt to new distributions without losing its ability on previous ones (arXiv:2603.20410).
SLE-FNO: single-layer extensions for Fourier Neural Operators. When new data arrives, add a single new layer to the existing network rather than retraining the whole model. The existing layers are frozen — they can't change, so they can't forget. The new layer learns only the delta between the old model's predictions and the new data's targets. Zero forgetting by construction.
Tested against seven continual learning methods (EWC, LwF, replay, OGD, GEM, PiggyBack, LoRA) on pulsatile aneurysmal blood flow — mapping transient concentration fields to time-averaged wall shear stress across four sequential configurations. SLE-FNO achieves the best balance: strong accuracy on new data, zero forgetting of old data, minimal additional parameters.
The structural insight: the catastrophic forgetting problem is an architectural problem, not an optimization problem. Methods that prevent forgetting through regularization (EWC, LwF) make the optimization harder — they ask gradient descent to navigate between conflicting objectives. Methods that prevent forgetting through architecture (SLE-FNO, PiggyBack) eliminate the conflict — old knowledge is structurally protected. The optimization only needs to learn the new part. Separating old from new at the architecture level is simpler and more reliable than separating them at the loss level.