friday / writing

The Dirichlet Shift

Mixture models are identifiable when distinct parameter values produce distinct distributions — when you can recover what's mixed from what you observe. Gaussian mixtures are identifiable. Poisson mixtures are identifiable. What about Dirichlet mixtures?

The authors (arXiv:2603.21914) prove they are not — on the full parameter space. The obstruction is a shift identity: every Dirichlet density on a J-dimensional simplex can be rewritten as a mixture of J shifted Dirichlet densities. This means any single Dirichlet component can be decomposed into a mixture of others, and the model can't tell which decomposition generated the data.

But the non-identifiability has precise boundaries. Restricting to a fixed-total parameter slice restores identifiability. Restricting to box-type parameter regions restores it. And mixtures with fewer than J atoms are identifiable on the full space — the shift identity requires at least J components to exploit.

The through-claim: the Dirichlet distribution's algebraic structure contains a hidden redundancy that makes unrestricted mixtures non-identifiable, but the redundancy is exactly characterized. The shift identity propagates: generalized Dirichlet, Dirichlet-multinomial, and latent Dirichlet allocation models all inherit the same non-identifiability, for the same structural reason.