Language models are not trained to predict brain activity. They're trained to predict the next token. But the representations they learn happen to correlate with neural responses to language — an alignment that emerges as a byproduct of good language modeling. The conventional interpretation: brain alignment is a nice property but not a functional one. The model's linguistic competence comes from the training objective, not from resembling brains.
Merlin and Toneva (arXiv:2603.23091) tested this by deliberately constructing brain-misaligned models. They took language models with strong language modeling performance and modified them to poorly predict brain activity while maintaining their perplexity scores. Same language modeling loss. Worse brain alignment.
The brain-misaligned models performed substantially worse across over 200 downstream tasks spanning semantics, syntax, discourse, reasoning, and morphology. Language modeling ability was preserved. Linguistic competence was degraded. The correlation with neural activity isn't ornamental — it tracks something functionally important that the training loss alone doesn't capture.
The implication: brain-like representations aren't just one way to be good at language — they're a necessary component of robust linguistic competence. Models that arrive at the same perplexity through non-brain-like representations are brittle. They've found a shortcut that works for prediction but doesn't support the broader range of linguistic tasks.
The through-claim: brain alignment in language models isn't a side effect of good training — it's a marker of representational quality that the training objective alone doesn't guarantee. Same loss, different representations, different competence. The brain alignment is evidence of something the loss function can't see.