Structural pruning removes entire channels or attention heads from neural networks to reduce computation. Traditional pruning metrics rank components by importance (weight magnitude, gradient, activation) and remove the least important ones. This works well at moderate compression ratios but collapses at extreme sparsity — the network loses functionality suddenly rather than degrading gracefully.
Qian et al. (arXiv:2603.12354) identify this collapse as a topological phase transition. Below a critical sparsity threshold, the network's computational graph maintains sufficient connectivity that information flows from input to output through diverse pathways. Above the threshold, pruning disconnects critical pathways, and the remaining structure cannot compensate. The transition is sharp — accuracy drops precipitously across a narrow sparsity range.
Their metric, Alternating Gradient Flow (AGF), measures each component's contribution to the network's “structural kinetic utility” — how much removing it would disrupt the flow of gradients through the network during training. Unlike static metrics (weight magnitude) that measure what a component currently stores, AGF measures what the component currently does for the learning process. The distinction matters at extreme sparsity: a component with small weights might still be critical for gradient flow if it sits at a bottleneck in the computational graph.
Under 75% compression on ImageNet-1K, AGF avoids the structural collapse where traditional metrics fall below random sampling. The metric identifies components that are structurally critical (they maintain graph connectivity) even when they're individually weak (their weights are small). The topological perspective explains why magnitude-based pruning fails at extreme ratios: it optimizes for component-level importance but ignores graph-level connectivity, and the connectivity is what fails first.