Neural audio codecs compress speech into discrete tokens. The tokens carry enough information to reconstruct intelligible audio. But where in the representation does accent live? Is it in which features activate, or in how strongly they activate?
Wang et al. decompose four codec models using sparse autoencoders and find that the answer depends on the codec's optimization target. Acoustic-oriented codecs encode accent primarily in activation magnitudes — the same features fire, but louder or softer depending on accent. Phonetic-oriented codecs encode accent in activation positions — different features fire entirely.
The same paralinguistic information, represented through two different mechanisms, emerging from two different training objectives. Acoustic codecs are optimized to reconstruct waveforms; they learn features that vary continuously, and accent shows up as a modulation of intensity. Phonetic codecs are optimized to capture linguistic content; they learn features that are categorical, and accent shows up as a different set of categories being activated.
A second finding: lower-bitrate codecs show stronger interpretability than higher-bitrate ones. Compression forces the representation to be efficient, and efficient representations are more structured — each feature must carry more meaning. The constraint of compression creates the interpretability that abundance destroys.
The structural lesson: how a system encodes information is determined by what it was optimized to do, not by the information itself. Accent is one thing in the world. In the codec, it is magnitude or position, continuous or categorical, depending entirely on the loss function that shaped the representation.