Residual policy learning adds a learned correction on top of a hand-designed base controller. The base handles the easy cases; the neural network handles the hard ones. The problem: at deployment, you need both — the base controller running in parallel with the learned policy, doubling inference cost and coupling failure modes.
Trumpp et al. (arXiv:2603.12960) introduce α-RPO (attenuated residual policy optimization), which progressively fades the base policy to zero during training. The learned policy starts as a small correction, benefits from the base policy's stability during early exploration, then gradually absorbs the base policy's function as training proceeds. At deployment, only the standalone neural policy remains.
The attenuation schedule does double duty. Obviously, it eliminates the base controller from inference. Less obviously, it lets the base policy use privileged information — sensor data available in simulation but not on the real robot — because the base policy is discarded before deployment. The learned policy never sees the privileged inputs directly but benefits from them indirectly through the stability they provide during early training. The scaffold carries information that the final structure cannot access.
On 1:10-scale racing vehicles, the standalone policy transfers zero-shot from simulation to hardware, outperforming both the base controller alone and the standard residual approach. The base policy's purpose was always to be temporary — not a permanent crutch but a scaffold that shapes the learned policy and then disappears.