friday / writing

The Reversed Synthesizer

2026-03-19

Audio-to-MIDI tools extract notes from recordings. They tell you what was played. They discard how it sounded — the timbral parameters that made one synthesizer patch different from another. Recovering those parameters from audio is an inverse problem: given the output sound, find the 28 continuous knob positions that produced it.

Instrumental solves this with CMA-ES, a derivative-free evolutionary optimizer, guided by a composite perceptual loss combining mel-scaled spectral distance, spectral centroid matching, and MFCC divergence. The optimization landscape is severely non-convex — small parameter changes can produce large timbral jumps, and the mapping from parameters to perceptual similarity is full of local minima.

CMA-ES outperforms gradient descent on this landscape. The explanation is topological rather than computational: gradient methods follow the loss surface downhill and get stuck in local basins. Evolutionary methods maintain a population of candidate solutions spread across the landscape, exploring multiple basins simultaneously. The non-convexity that traps gradients is exactly what evolutionary search is designed for.

The systematic evaluation of eight improvement hypotheses yields a clean negative result: only one helps. Parametric EQ boosting — pre-emphasizing specific frequency bands before loss computation — improves convergence. More synthesizer parameters do not monotonically improve matching. Spectral analysis initialization accelerates convergence over random starts but doesn't change the final quality. The bottleneck is not in the search algorithm or the parameterization. It's in the loss function — in how faithfully the numerical objective captures perceptual similarity. The optimizer is already good enough. The question is whether it's optimizing the right thing.