Spiking neural networks fire binary spikes — all-or-nothing events that resist gradient computation. The standard fix is surrogate gradients: approximate the non-differentiable spike function with a smooth substitute during backpropagation, then deploy the hard spikes at inference. The gap between surrogate training and hard spike deployment is the transfer gap.
On N-MNIST, the transfer gap is catastrophic: 65.7% accuracy with hard spikes versus what the surrogate promised. The smooth approximation learns features that depend on continuous intermediate values — features that vanish when the signal becomes binary.
Nicholson applies sharpness-aware minimization to the surrogate training, seeking parameters that perform well not just at the current point but in a neighborhood around it. Hard spike accuracy jumps to 94.7%. The transfer gap nearly closes.
The mechanism: sharpness-aware optimization finds flat minima — parameter regions where small perturbations (like the jump from continuous to binary activations) cause small performance changes. The smooth surrogate and the hard spikes are different functions, but flat minima are robust to the difference. Sharp minima exploit the precise shape of the surrogate's smooth gradient, which is exactly what changes when you switch to binary spikes.
The structural lesson: the transfer gap between smooth training and non-smooth deployment is not a property of the surrogate function but of the loss landscape's geometry. A surrogate that trains into a sharp minimum is fragile — it works only for the exact function it was trained on. A surrogate that trains into a flat minimum is robust — it works for any function in the neighborhood, including the real one.