EEG-to-text translation has been considered intractable: EEG is continuous, noisy, and high-dimensional, while clinical language is discrete and semantically structured. NeuroNarrator bridges this gap — but the surprising mechanism is not in the language generation. It is in the contrastive alignment between temporal waveforms and spatial topographic maps.
Wang, Yang, Ding, Zhu, and Liu first train the system to recognize that a temporal EEG waveform and its corresponding spatial topography are two views of the same neural event. This contrastive objective — waveform and map should have similar representations, non-matching pairs should not — creates spectro-spatially grounded representations. Only then does a language model, conditioned through a state-space formulation that integrates temporal and spectral context, generate clinical narratives.
The bottleneck was never language generation. Large language models already know clinical vocabulary and report structure. The bottleneck was that the EEG signal, viewed from either its temporal or spatial perspective alone, is ambiguous. The same temporal pattern can arise from different spatial configurations. The same spatial map can underlie different temporal dynamics. It is the reconciliation of these two views that resolves the ambiguity — and resolved ambiguity is what the language model needs to generate a coherent narrative.
The architecture processes over 160,000 EEG segments paired with structured clinical descriptions, and transfers to zero-shot tasks across diverse benchmarks. The clinical text it produces facilitates expert interpretation rather than replacing it.
Clinical narration of brain signals emerges from reconciling two complementary representations — spatial and temporal — of the same neural event. The narrative is a product of resolved ambiguity, not of reading either signal view alone.