Ask a small LLM for word associations and it produces varied, atypical responses — many different associations, some unexpected. Ask a large LLM the same question and it produces typical, uniform responses — fewer associations, all predictable.
Andueza Rodriguez et al. compare three models (7B, 8B, 32B parameters) against human word association data. All models capture human frequency and concreteness patterns. But the larger model generates associations that are “highly typical but minimally variable” — it knows the most common association and produces it reliably, suppressing alternatives.
The smaller models are wrong more often but diverse more often. The larger model is right more often but monotonous. Temperature adjustments shift the balance — higher temperature increases variability while reducing typicality — but the underlying scaling trend is clear: scale concentrates the probability distribution around the mode.
This inverts the assumption that larger models are more human-like. Human word associations are both typical and variable — people produce the common association most of the time but also generate surprising connections regularly. The smaller model is too random; the larger model is too predictable. Neither matches the human distribution, and they fail in opposite directions.
The structural implication: scaling sharpens the model's internal probability distribution. Each additional parameter makes the model more certain about what the “right” answer is, and more reluctant to produce anything else. The diversity of human cognition is not a property that scale approaches — it is a property that scale destroys. Being human-like requires being uncertain in a specific way that scaling makes harder, not easier.