Three variational autoencoders for musical timbre — unsupervised, descriptor-conditioned, perceptual-feature-conditioned — organize their latent spaces in fundamentally different ways. The comparison reveals what kind of conditioning information actually structures a generative model's internal representation.
The unsupervised VAE discovers its own axes. Without guidance, it finds whatever statistical regularities minimize reconstruction error. The resulting latent space is functional but uninterpretable — sounds that seem similar to humans may land far apart, and nearby points may differ in perceptually irrelevant ways. The model optimizes for its own objective, not for human categories.
Descriptor conditioning uses one-hot labels: “bright,” “warm,” “harsh.” Each descriptor gets a discrete category. The latent space improves — sounds with the same label cluster together. But the conditioning is coarse. A sound labeled “bright” at intensity 2 and one at intensity 4 receive the same one-hot vector. The gradient between categories is invisible to the model. Discrete labels create discrete clusters rather than continuous manifolds.
Perceptual-feature conditioning uses continuous values from timbral analysis models — numerical measurements of brightness, warmth, roughness. The latent space becomes compact, discriminative, and pitch-invariant. Sounds separate cleanly by timbre regardless of fundamental frequency. The continuous conditioning creates continuous structure.
The lesson is about the information format, not the information content. Descriptor labels and perceptual features encode roughly the same knowledge about timbre. But continuous values allow the model to learn gradients — the relationship between “slightly bright” and “very bright” — while discrete labels only distinguish categories. The latent space reflects the structure of its conditioning signal, not just its content.