Data-driven weather forecasting has reached accuracy competitive with operational numerical systems. But the training cost is enormous: current models require massive GPU clusters, putting them beyond the reach of most research groups.
Sonny (arXiv: 2603.21284) is a hierarchical transformer that achieves competitive medium-range forecasting while training on a single NVIDIA A40 GPU in 5.5 days.
The architecture has two stages. A narrow “slow path” first models large-scale atmospheric dynamics — the planetary waves, jet streams, and pressure patterns that set the background. A full-width “fast path” then integrates thermodynamic interactions — moisture, temperature gradients, convection — on top of the large-scale skeleton. The hierarchy lets the expensive attention mechanism focus on the large-scale structure where it matters most.
Exponential moving average (EMA) during training stabilizes multi-step rollout without requiring a separate fine-tuning stage. On WeatherBench2, Sonny is competitive with operational baselines and outperforms FastNet at extended tropical lead times — where thermodynamic interactions dominate over large-scale dynamics.
The through-claim: weather forecasting's compute wall is architectural, not fundamental. The atmosphere has hierarchical structure — large scales set the stage, small scales fill in details — and an architecture that matches this hierarchy is dramatically more efficient than a flat one. The physics suggests the architecture; the architecture reduces the compute.
2603.21284. Atmospheric science / weather forecasting / deep learning / transformers / computational efficiency.