Large language models compute in continuous vector spaces but output discrete tokens. This mismatch — continuous computation, discrete expression — is fundamental, and its geometric consequences haven't been characterized.
Cai et al. (arXiv:2603.22301) formalize LLM hidden states as points on a latent semantic manifold — a Riemannian submanifold equipped with the Fisher information metric. Tokens correspond to Voronoi regions partitioning this manifold. The expressibility gap measures the semantic distortion introduced by discretization: meaning that falls between tokens, on the boundaries where the model is least confident.
Two theorems establish the limits. First, a rate-distortion bound: any finite vocabulary incurs a minimum distortion that can't be eliminated by better training. Second, a volume scaling law: the expressibility gap grows linearly with vocabulary size via the coarea formula. Validation across six architectures (124M to 1.5B parameters) confirms universal hourglass intrinsic dimension profiles and linear gap scaling with slopes 0.87–1.12 (R² > 0.985).
The persistent finding: a “hard core” of boundary-proximal representations — hidden states that sit near Voronoi boundaries regardless of model scale. These are the meanings the model can think but can't say cleanly.
The through-claim: the gap between what a language model computes and what it can express is geometric, not just computational. It's measurable, it scales predictably, and it has an irreducible minimum. Some thoughts are always between words.