Diffusion models generate high-quality images but require many sequential denoising steps — each step is a full network evaluation. Distillation methods compress the trajectory into fewer steps but require retraining the entire model, which is expensive and sometimes degrades quality.
The authors (arXiv:2603.22375) find that the bottleneck for few-step sampling isn't the network weights — it's the timestep conditioning. Standard diffusion models use a single time embedding per step, but different layers denoise at different rates and scales. When you reduce from 50 steps to 3, this mismatch becomes critical: each of the 3 steps must simultaneously handle what previously took many specialized steps, and a single time embedding can't coordinate the diverse layer-level requirements.
Multi-layer Time Embedding Optimization (MTEO) freezes the pretrained backbone entirely and optimizes only a small set of per-step, per-layer time embeddings distilled from reference trajectories. The embeddings tell each layer exactly where it is in the compressed denoising schedule. No architecture changes, no inference overhead, a tiny fraction of parameters trained.
Three steps achieve state-of-the-art few-step generation across diverse datasets and architectures.
The through-claim: the diffusion model already knows how to denoise — it just doesn't know when it is. Compressing the sampling schedule creates a coordination problem, not a capacity problem. Fixing the clock at each layer is cheaper and more effective than retraining the entire model.