Global pooling averages an embedding vector across all positions in a sequence. For text, this collapses a sentence into a single vector that captures its overall meaning. For genomic sequences, it collapses an entire chromosome into a single representation. In both cases, position is discarded.
Cross-species antimicrobial resistance prediction (arXiv:2603.11141) reveals why discarding position is catastrophic for this specific problem. Resistance mechanisms are carried by horizontally transferred gene cassettes — mobile genetic elements that insert at specific genomic locations. The resistance signal is not distributed across the genome. It is concentrated in islands. Global pooling averages the resistance islands with megabases of irrelevant background, diluting the signal below detectability.
The fix is MiniRocket applied to per-window embeddings — treating the genome as an ordered multivariate time series rather than a bag of features. This preserves local activation patterns: spikes in the embedding that correspond to resistance cassettes. Foundation model embeddings from deeper layers, selected near a stability boundary, provide the most robust per-window representations.
The standard technique (global pooling) fails because the biology has structure the technique ignores. Resistance genes didn't evolve in their current hosts. They were imported — carried in on mobile elements from distant species. Their genomic context is foreign. A pooling operation that treats all positions equally treats the foreign insert the same as the native background, which is precisely the wrong thing to do when the foreign insert is the entire signal.
Within-species k-mer baselines perform well — the same cassettes appear in the same genomic contexts. But under true cross-species evaluation, k-mer performance collapses. The patterns that predict resistance in one species don't transfer because the surrounding genome differs. Only the localized activation patterns survive the species boundary, because only they correspond to the horizontally transferred elements that actually carry resistance.
The aggregation strategy matters more than model scale. The representation is fine. The summary destroys it.