A deep reinforcement learning policy trained to make a simulated cheetah run has no explicit notion of gait phases. It receives joint angles and velocities, outputs motor torques, and maximizes forward speed. There are no stance and swing phases in the reward function, no flight phases in the training curriculum, no biological priors in the architecture. The policy is a neural network mapping observations to actions.
It invents gait phases anyway. Analysis of the trained policy reveals periodic phase transition structures — the state sequences naturally cluster into distinct groups with consistent internal dynamics and predictable transitions between them. The clusters correspond to recognizable phases of locomotion: stance, swing, flight. More surprisingly, the policy exhibits phase branching — at certain states, different actions are taken depending on subtle features of the configuration, creating multiple possible phase trajectories from the same approximate state.
The phase-dependent strategy is not uniform. Explainable Boosting Machines reveal that the policy prioritizes different state features in different phases. During stance, leg configuration dominates. During flight, body orientation matters more. The policy has learned to pay attention to different things at different times — a context-dependent attention strategy that mirrors biological motor control.
The structural point: gait phases are not a design choice that must be imposed. They are an emergent organizational structure that arises because periodic locomotion requires periodic control, and periodic control on a high-dimensional system naturally decomposes into phases with distinct dynamics. The policy discovers this structure for the same reason animals evolved it — it is the natural way to organize a cyclic task with multiple competing requirements.
(arXiv:2603.18084)