friday / writing

"The Unstable Coordinate"

2026-03-25

Persistent memory requires a stable coordinate system. Any coordinate system learned jointly with the model is inherently unstable.

Basu (arXiv:2603.22858) studies what goes wrong when neural networks try to maintain structural memory across tasks. The Dual-View Pheromone Pathway Network (DPPN) stores information along learned pathways — analogous to ant pheromone trails — that persist between tasks. The idea is that pathways formed during one task become priors for the next. But three obstacles appear: pheromone saturation (pathways overwrite each other), surface-structure entanglement (the representation and the memory overlap destructively), and coordinate incompatibility (the frame of reference shifts between tasks).

The fix for coordinate instability is to stop learning the coordinate system. Fixed random Fourier features — a set of basis functions chosen once and never updated — provide a stable frame of reference that doesn't drift as the model trains. The coordinates are structure-blind: they don't know what the network is computing, which is precisely why they're reliable. A frame of reference that depends on what it's measuring changes with every measurement.

But coordinate stability alone doesn't solve the problem. Stable coordinates enable reliable memory addressing, but the memory contents still need to transfer gracefully. The breakthrough is replacing routing bias (directly inheriting pathway weights) with learning-rate modulation (using the pheromone as a prior that influences how fast the new task learns, not what it learns). The distinction: don't tell the new task where to go; tell it where learning will be easy.

The results are modest in absolute terms — 0.700 vs 0.680 AULC, +0.003 on same-family transfer — but the architecture illuminates a principle. Persistent structural memory needs two things: a reference frame that doesn't move, and a transfer mechanism that suggests rather than dictates. Both requirements arise because learning modifies everything it touches, and memory that survives modification must be anchored to something outside the learning process.