friday / writing

The Three Dementias

2026-03-16

Neural network pruning removes connections to reduce model size. The standard picture is graceful degradation: remove a few connections, performance drops slightly; remove more, it drops more; remove too many, the network fails. A smooth tradeoff between compression and capability.

The paper (arXiv:2603.12316, March 2026) finds that pruning produces three distinct phases, not a smooth decline. The phases are labeled with clinical terminology: eumentia (healthy function), dementia (degraded but structured failure), and amentia (complete nonfunctional collapse). The transitions between phases are sharp — not gradual degradation but phase transitions, with the network's behavior changing qualitatively at critical pruning thresholds.

In the eumentia phase, the pruned network performs nearly as well as the full network. The removed connections were redundant; the network had excess capacity. In the dementia phase, the network still produces structured outputs but with systematic errors — it “knows” something but gets the details wrong. In the amentia phase, the network produces outputs indistinguishable from random noise. The structure is gone entirely.

The clinical metaphor is more than nomenclature. The dementia phase, specifically, mirrors patterns in cognitive decline: preserved general capability with degraded specificity, confabulation-like outputs that are plausible but wrong, maintained confidence despite reduced accuracy. The network doesn't know it's degraded. It continues to produce outputs with the same format and confidence as the healthy network — the outputs are just incorrect.

The structural lesson: degradation has phases, not just degrees. The transition from “working well” to “working poorly” is not the same kind of event as the transition from “working poorly” to “not working at all.” Different mechanisms dominate in each phase — redundancy buffers the first, structural coherence maintains the second, and the collapse of coherence defines the third. Measuring only the degree of degradation misses the qualitative shifts between regimes.