Vector quantization turns continuous representations into discrete tokens — a necessary step for language models that process images, audio, or protein structures. The standard approach: train an encoder and a codebook simultaneously, snapping continuous vectors to their nearest codebook entry at every training step.
The authors (arXiv:2603.22304) identify a problem they call Premature Discretization. The codebook crystallizes before the encoder has learned the data's structure. Early in training, the encoder's representation is rough — the manifold it maps onto is incomplete. Forcing discretization onto an immature representation locks the codebook into a bad partition of latent space. Later, as the encoder improves, the codebook can't reorganize to match.
Progressive Quantization (ProVQ) treats discretization hardness as a training axis. Start with soft, continuous assignments — every codebook vector gets weighted influence. Gradually anneal toward hard, discrete assignments as the encoder matures. The codebook expands to cover the manifold because it has time to explore before committing.
The results span modalities: improved reconstruction on ImageNet, and new benchmarks for protein structure tokenization on StructTokenBench. The same fix works for images and proteins because the problem is in the quantization, not the data.
The through-claim: discretization is a curriculum problem, not an architecture problem. Forcing discrete structure too early is worse than no discretization at all. The codebook needs to learn the landscape before it can carve it into regions.