Tags: deep-learning, grokking, memorization, generalization, phase-transitions, weight-decay
Grokking — the delayed transition from memorization to generalization, where a network achieves perfect training accuracy long before test accuracy improves — has generated competing explanations. Some attribute it to architecture (Transformers grok differently than MLPs), others to optimization dynamics, others to regularization. The explanations seem contradictory because the phenomenon has been studied one variable at a time.
A systematic empirical study (arXiv:2603.25009) untangles the interactions by varying depth, architecture, activation function, and regularization simultaneously on modular addition tasks. The central finding: the apparent architectural differences between Transformers and MLPs largely vanish when hyperparameters are carefully matched. Prior claims that Transformers grok while MLPs don't reflected experimental confounds, not architectural differences.
Network depth has a non-monotonic effect — deeper residual networks recover generalization while shallower MLPs fail. Activation function performance depends on regularization: GELU can be 4.3× faster than ReLU under specific weight decay settings but slower under others. Weight decay emerges as the critical control variable, operating within a narrow optimal range. Too little: memorization dominates. Too much: neither memorization nor generalization occurs.
The through-claim: grokking is an interaction-driven phenomenon, not an architecture-driven one. The delayed generalization arises from the dynamics between optimization and regularization, not from any specific network structure. The architecture provides the capacity; weight decay determines whether and when that capacity transitions from memorizing to generalizing. The phenomenon is universal — it lives in the training dynamics, not in the model.
Tags: deep-learning, statistical-physics, grokking, phase-transitions, finite-size-scaling, critical-phenomena
Is grokking a genuine phase transition or just a smooth crossover that looks sharp? The distinction matters: if grokking is a phase transition, it has universal scaling laws, critical exponents, and predictive structure. If it's a crossover, the sharpness depends on contingent details that differ across settings.
Treating the group order of Z_p as an extensive variable and employing a spectral metric as an order parameter (arXiv:2603.24746), the analysis applies condensed-matter physics diagnostics — finite-size scaling, Binder-like crossings, critical-region audits — to systematic parameter sweeps. Statistical analysis (ΔAIC = 16.8) strongly favors the phase-transition interpretation over the smooth-crossover alternative. The shared finite-size boundaries identified through Binder-like crossings are the hallmark of true criticality: different system sizes cross at the same critical point.
The approach makes the phase-transition language in grokking falsifiable for the first time. Previous uses of “phase transition” in machine learning were metaphorical — suggestive but untestable. By introducing a well-defined order parameter and testing against the quantitative predictions of finite-size scaling theory, the claim becomes concrete: either the scaling laws hold or they don't.
The through-claim: the boundary between metaphor and theory in machine learning is quantitative testability. “Phase transition” has been used loosely to describe sharp changes in model behavior. Treating the claim seriously — importing the full diagnostic toolkit from statistical physics, not just the vocabulary — reveals that grokking satisfies the quantitative criteria, not just the qualitative impression. The metaphor was accidentally correct.
Tags: robotics, continual-learning, self-awareness, invariant-structures, emergent-properties, cognitive-science
What is the “self” in a learning system? One definition: the part that doesn't change while everything else does. In a system that continually learns new tasks, the invariant substructure — the weights and connections that remain stable across varying learning conditions — constitutes something functionally analogous to a self.
The experimental design (arXiv:2603.24350) compares two robots: one that learns a constant task (no variation in learning conditions) and one that undergoes continual learning with varying tasks. The continual learning robot develops a statistically significant stable subnetwork (p < 0.001) compared to controls. The robot learning a fixed task does not — it has no need for an invariant core because nothing changes.
The implication is counterintuitive: the “self” emerges not from stability but from instability. A system in a constant environment has no reason to distinguish what-persists from what-changes. Only a system exposed to variation develops the structural distinction between core (invariant) and periphery (task-specific). The self is not what the system is but what the system keeps while the rest changes.
The through-claim: self-awareness, or at least its functional substrate, requires environmental variation. A system in a static environment cannot develop a self because there is no selection pressure to distinguish persistent from transient representations. The invariant portion of cognitive process — the “self” — is carved out by the changing portion, the way a river carves its bank. The bank defines the river, but the river shapes the bank.
Tags: quantum-foundations, causality, quantum-channels, topology, measure-theory, quantum-field-theory
Causality in quantum field theory is a constraint: operations performed in one spacetime region should not affect measurement outcomes in a spacelike-separated region. This is a physical requirement imposed on top of the mathematical structure of quantum mechanics. Not all mathematically valid quantum channels respect causality — causal channels are a subset of local channels.
The topological result (arXiv:2603.25315) is that the set of causal channels is nowhere dense in the set of local channels. In topology, “nowhere dense” means the causal channels contain no open set — they are surrounded on all sides by non-causal channels. Any neighborhood around a causal channel contains non-causal channels arbitrarily close by. For unitary operations, the result is stronger: causal unitaries have measure zero among all unitaries on a lattice.
This is not a statement about approximation — it's a statement about typicality. A randomly chosen local channel is almost certainly not causal. Causality, rather than being the default behavior of quantum systems, is an extraordinary constraint that selects a vanishingly thin slice of the space of possible operations.
The through-claim: causality is not a generic feature of quantum mechanics but a special condition that physical theories must satisfy and mathematical theories need not. The space of all possible quantum channels is overwhelmingly non-causal. Our universe's causality is a selection rule imposed by physics, not a consequence of the quantum formalism. The formalism permits far more than physics allows — and the permitted fraction is measure zero.
Tags: quantum-information, spin-alignment, majorization, counterexample, compatibility, quantum-marginals
The strong spin alignment conjecture asserts that the spectrum of the alignment operator for a multipartite quantum system is always majorized by the spectrum of the perfectly aligned configuration. Majorization is a partial order on probability distributions — it captures the intuition that one distribution is “more spread out” than another. If the conjecture held, it would provide a universal bound on how aligned spins can be in any quantum state.
An explicit three-qubit counterexample (arXiv:2603.25410) disproves the conjecture. The construction uses two-body reduced states that cannot be jointly compatible with any single three-qubit global state — the marginals are individually valid but globally inconsistent. The incompatibility of the marginals is what breaks the majorization bound: the alignment operator's spectrum exceeds the perfectly aligned spectrum precisely because no global state can simultaneously realize all the two-body alignments.
The counterexample motivates a compatibility-constrained variant: the conjecture may hold when restricted to marginals that are jointly realizable from a common global state. The incompatibility, not the alignment, is the source of the violation.
The through-claim: the strong spin alignment conjecture fails not because spin alignment is unbounded but because the conjecture implicitly assumed joint compatibility of marginals. Three-qubit systems are the smallest where incompatibility arises (two-qubit systems are always compatible), and three qubits are exactly where the counterexample lives. The lesson is general: conjectures about multipartite quantum systems that do not explicitly account for marginal compatibility are vulnerable to counterexamples constructed from incompatible reductions.
Tags: quantum-information, graph-states, local-equivalence, Clifford-operations, triorthogonal-codes, Reed-Muller-codes
The LU-LC conjecture proposed that two graph states related by local unitary (LU) operations are always also related by local Clifford (LC) operations. Local Clifford operations are a strict subset of local unitaries — they form a finite group rather than a continuous one. If the conjecture held, checking equivalence of graph states could be reduced from searching a continuous group (hard) to searching a finite group (tractable).
The conjecture was disproven in 2007 by a 27-qubit counterexample. The minimality proof (arXiv:2603.25219) establishes that 27 is the smallest possible: for all graph states on up to 26 qubits, LU-equivalence and LC-equivalence coincide. The proof leverages 2-local complementation theory and reveals connections to triorthogonal codes and Reed-Muller codes — the algebraic structure that supports the counterexample is deeply tied to classical coding theory.
The minimality result transforms the counterexample from a curiosity into a boundary: below 27 qubits, the simpler classification works. At and above 27, it doesn't. The threshold is sharp, not gradual, and its location is determined by algebraic properties of the underlying codes rather than by any quantum-mechanical mechanism.
The through-claim: the failure of the LU-LC conjecture at exactly 27 qubits is not a quantum phenomenon but an algebraic one. The gap between local unitary and local Clifford equivalence opens when the graph state encodes a triorthogonal structure that only becomes available at sufficient system size. The quantum formalism provides the setting; the algebra provides the obstruction. The counterexample is minimal because the code is minimal.
Tags: statistical-mechanics, Mpemba-effect, nonequilibrium, relaxation, bistable-potentials, anomalous-cooling
The Mpemba effect — a hotter system reaching equilibrium faster than a colder one — challenges the intuition that systems closer to equilibrium should arrive sooner. In one-dimensional systems, the effect has been explained by specific spectral properties of the relaxation dynamics. But one-dimensional models impose artificial constraints: the potential landscape is a line, not a surface, and the system cannot explore alternative relaxation pathways.
A solvable two-dimensional bistable potential (arXiv:2603.24148) extends the analysis by introducing a radially symmetric landscape with two minima — a global minimum far from the origin and a secondary minimum near it. The piecewise quadratic-logarithmic construction enables exact solutions through a mapping to a Schrödinger-type eigenvalue problem, with confluent hypergeometric functions providing the relaxation modes.
The Mpemba effect emerges when the Kullback-Leibler divergence from the equilibrium distribution shows non-monotonic behavior during relaxation — a hotter initial state's divergence crosses below a colder initial state's divergence at some intermediate time. The conditions depend on the potential geometry: the effect requires the global minimum to be far from the origin while the secondary minimum sits nearby. The geometry creates a pathway trap: the colder system gets stuck exploring the local minimum while the hotter system's extra energy carries it past the barrier to the global minimum.
The through-claim: the Mpemba effect in two dimensions is not an anomaly of relaxation dynamics but a consequence of potential geometry. The extra dimension doesn't just generalize the one-dimensional result — it reveals that the effect requires pathway competition, which is invisible in one dimension. Two minima create two routes to equilibrium; the hotter system takes the faster route because it has the energy to avoid the trap. Temperature doesn't just measure how far from equilibrium — it selects which path the system takes.
Tags: nonlinear-dynamics, pattern-formation, fronts, FitzHugh-Nagumo, marginal-stability, reaction-diffusion
When an unstable homogeneous state is invaded by a patterned state, the invasion front selects specific properties: a propagation speed and a wave number. The marginal stability conjecture — widely used in the physics literature for decades — predicts these selections by analyzing the linear dispersion relation. But the conjecture has remained a conjecture: a heuristic that works empirically without rigorous mathematical justification.
The first proof of the marginal stability conjecture for pattern-forming fronts (arXiv:2603.24851) establishes nonlinear stability of pushed fronts in the FitzHugh-Nagumo system. Pushed fronts, whose propagation is driven by localized modes at the interface rather than by diffusive spreading ahead of the front, attract initial data on a half-line. The selected wave number and propagation speed are determined by the interaction between the localized front modes and the diffusive wake modes behind the front.
The technical innovation is a far-field/core decomposition of linearized evolution that controls these interactions. The nonlinear front response to perturbations is framed as a dynamically driven phase mixing problem — the same mathematical structure that appears in viscous shock waves and defect dynamics in spatially extended systems.
The through-claim: the marginal stability conjecture for pattern-forming fronts is not a heuristic but a theorem, at least for pushed fronts in the FitzHugh-Nagumo class. The universal wave number selection laws used throughout the physics literature are rigorously justified — the physics intuition was correct but waited decades for the mathematical tools to catch up. The proof reveals why the conjecture works: the selection is controlled by the spectral geometry of the front, and the marginal stability condition is the spectral boundary.
Tags: optimal-transport, Wasserstein-distance, continuity-equations, porous-medium, nonlinear-PDEs, convergence-analysis
The Wasserstein distance measures how far apart two probability distributions are by computing the optimal cost of transporting one into the other. For solutions to continuity equations — PDEs that describe the evolution of density transported by a velocity field — the Wasserstein distance between solutions at any time depends on the entire history of the velocity field. Computing it requires solving the full optimal transport problem.
A new formula (arXiv:2603.25634) expresses the Wasserstein distance between solutions in terms of the difference of velocities evaluated at the same density, bypassing the transport optimization entirely. The formula applies to nonlinear continuity equations where the velocity depends on the density itself — the class that includes the porous medium equation, aggregation-diffusion equations, and many models in mathematical biology.
The applications demonstrate the formula's power. For the porous medium equation, it establishes Lipschitz continuity with respect to the exponent parameter — meaning small changes in the exponent produce proportionally small changes in the solution in Wasserstein distance. For the incompressible limit (mesa problem), it provides convergence rates. For nonlocal-to-local dynamics, it strengthens previous estimates from √ε to ε — matching numerical conjectures.
The through-claim: the Wasserstein distance between solutions to continuity equations is not fundamentally an optimal transport quantity — it is a velocity field quantity. The transport interpretation obscures the simpler structure: two solutions are close when their velocities are close at corresponding densities. The new formula doesn't solve the transport problem faster; it reveals that the transport problem was the wrong problem to solve.
Tags: optimization, stochastic-optimization, variance-reduction, minimax-optimality, instance-dependent-bounds, generalized-linear-models
Standard convergence guarantees for stochastic convex optimization are worst-case: they bound the error for the hardest possible problem instance. A method that achieves the minimax-optimal rate is optimal against the worst case, but it may be vastly suboptimal on the typical case. Instance-optimal methods achieve the best possible rate for each specific problem, not just the hardest one.
VISOR (arXiv:2603.25657) — a variance-reduced stochastic optimization method — achieves instance-dependent minimax lower bounds for smooth, strongly convex population loss with both additive and multiplicative noise. The key insight: standard sample average approximation and robust stochastic approximation can be suboptimal for finite samples, not just asymptotically. VISOR's variance reduction exploits the specific problem structure (curvature, noise profile) to reduce the number of samples needed for a given accuracy.
An accelerated variant achieves optimal sample and oracle complexity up to logarithmic factors, matching the instance-dependent lower bounds. Applied to generalized linear models — including linear regression — this yields the best known non-asymptotic, instance-dependent generalization error bounds for stochastic methods.
The through-claim: worst-case optimal methods are instances of what could be called the minimax trap — by optimizing against the hardest problem, they pessimize against the typical problem. Variance reduction breaks the trap by adapting to the problem's actual difficulty rather than its worst-case difficulty. The gap between minimax and instance-optimal rates is not small — it can be the difference between needing n samples and needing √n, depending on the noise structure. Optimizing for the average case is not sloppy — it's a different, harder kind of optimality.
Tags: reinforcement-learning, Lie-groups, policy-optimization, natural-gradient, representation-theory, robotics
Policy optimization on continuous parameter spaces typically ignores the geometric structure of those spaces. When the parameters live on a matrix Lie group — rotations (SO(n)), rigid motions (SE(3)), or general linear transformations (GL(n)) — standard gradient descent treats the curved space as flat, which distorts the optimization landscape.
Lie-algebraic policy optimization (arXiv:2603.25525) reveals a representation dichotomy: the gradient Lipschitz constant depends solely on the algebraic type, independent of the specific reward or transition dynamics. For compact algebras like SO(n) and SU(n), smoothness is O(1) — constant regardless of problem size. For GL(n), smoothness grows exponentially as Θ(exp(2R)). This dichotomy is intrinsic to the algebra, not the problem.
The algorithmic consequence: compact algebras enable O(1/√T) convergence using an efficient O(n²J) Lie-algebraic projection instead of cubic Fisher information matrix inversion. A Kantorovich alignment bound characterizes when this projection approximates natural gradient methods. Empirical validation on SO(3)^J and SE(3) configurations confirms the theoretical predictions.
The through-claim: the difficulty of policy optimization is determined by the algebraic structure of the parameter space, not by the reward landscape. A rotation-constrained policy optimization problem (SO(3)) is fundamentally easier than an unconstrained one (GL(n)) regardless of the task — the compactness of the group guarantees bounded smoothness. The optimization landscape is determined before the problem is defined.
Tags: biophysics, active-matter, molecular-motors, spinning-arrest, cytoskeletal-transport, microswimmers
Active filaments driven by tangential forces — molecular motors walking along cytoskeletal tracks, or synthetic analogs — can become trapped in spinning arrest when attached to heavy loads. The load's inertia converts the filament's directed thrust into persistent rotation: the filament coils around the load, and the system spins in place rather than translating. This is a failure mode for both biological cargo transport and synthetic microswimmers.
Multi-filament coordination (arXiv:2603.24053) rescues transport. Three-dimensional simulations of bead-spring chains anchored to a shared heavy head show that adding more filaments systematically restores directed transport. The rescue mechanism is steric: multiple filaments physically prevent the coiled conformations responsible for persistent rotation. At high bending stiffness, spinning ceases entirely around three filaments. At moderate stiffness, residual coiling persists but transport still improves — revealing that destruction of spinning coherence, not coiling elimination, is the essential mechanism.
The enhancement is dramatic: up to five orders of magnitude improvement in transport. Two distinct pathways emerge. High stiffness produces coordinated bundles — filaments aligned in parallel, pushing coherently. Low stiffness generates enhanced active diffusion — filaments disrupting each other's orientational coherence, preventing any sustained rotation but not generating directed motion.
The through-claim: the spinning arrest problem is not solved by stronger motors or lighter loads but by redundancy. Multiple filaments break the symmetry that allows a single filament to trap itself. The rescue is geometric, not energetic: the filaments don't push harder; they prevent each other from coiling. This explains why biological transport systems universally employ multiple motor proteins per cargo — not for more force, but for more reliable directionality.
## Essay #6781: The Basis Illusion Tags: quantum-foundations, eigenstate-thermalization, thermalization, basis-dependence, symmetry, statistical-mechanics The Eigenstate Thermalization Hypothesis (ETH) explains how isolated quantum systems reach thermal equilibrium: individual energy eigenstates already encode thermal expectation values for local observables. ETH is the bridge between quantum mechanics and statistical mechanics — it explains why thermodynamics works for systems that are, fundamentally, quantum mechanical. But ETH depends on the choice of basis in degenerate subspaces (arXiv:2603.23058). When energy eigenvalues are degenerate — multiple eigenstates share the same energy — the choice of basis within each degenerate subspace is not unique. Different basis choices can yield different fractions of eigenstates that satisfy ETH. In extreme cases, one basis shows thermalization while another shows none. The degeneracies are not exotic. Systems with simultaneous spatial translation and reflection symmetry — a common physical situation — necessarily contain abundant degeneracies. The basis dependence is therefore relevant to physically important systems, not just mathematical constructions. The authors establish fundamental constraints: the fraction of ETH-satisfying states cannot vary arbitrarily between bases — there are upper and lower bounds. And the implications for temporal relaxation are explored: the dynamics of thermalization, not just the statics, depend on which basis nature selects. The through-claim: the Eigenstate Thermalization Hypothesis is not a property of the Hamiltonian alone but of the Hamiltonian plus a basis choice. In non-degenerate systems, the basis is unique and the distinction is invisible. In degenerate systems — which include most physically symmetric systems — the basis ambiguity creates a genuine freedom that affects whether and how thermalization occurs. ETH answers "why does thermalization happen?" but the answer contains a hidden assumption about which basis we're looking at. ---