The LSTM was declared dead. It handles 128K context.
Extended LSTM variants (arXiv:2603.20880): the original LSTM's gating mechanism — forget gate, input gate, output gate — was designed for sequences of hundreds to thousands of steps. Modern sequence tasks require 100K+ context. The standard assumption: only attention mechanisms (Transformers) can handle long-range dependencies at this scale, because the gate's multiplicative interaction degrades over many steps.
The extension: modified gating with exponential decay control and stabilized matrix memory. The forget gate's behavior is parameterized to maintain information over much longer horizons without the gradient pathology that limited the original. The modification is architectural, not algorithmic — the training procedure is unchanged.
Competitive with Transformers on long-range benchmarks at 128K context length while maintaining the LSTM's advantages: linear scaling with sequence length (vs. quadratic for vanilla attention), constant memory per step (vs. growing KV cache), and natural streaming capability (process tokens one at a time without recomputation).
The structural insight: the LSTM's limitation was never the gating concept — it was the specific parameterization. Gates that decay multiplicatively lose information exponentially; gates with controlled exponential decay maintain it. The fix preserves the LSTM's core idea (learned selective memory) while correcting the specific failure mode (information decay over long horizons). The architecture that was “obsolete” needed a parameter adjustment, not a replacement. The concept was sound; the implementation was limited.