Minimizing a divergence subject to constraints is the engine behind hypothesis testing, estimation, and change detection. You want the distribution closest to a reference — in the KL sense or some other f-divergence — that satisfies given moment conditions. The dual formulation converts this infinite-dimensional optimization (over distributions) into a finite-dimensional one (over Lagrange multipliers).
The paper on dual representation of minimum divergence (arXiv: 2603.21027) develops a systematic method for deriving these dual representations on compact domains.
The approach is two-stage: first solve the finite-dimensional problem (distributions supported on finitely many points), where convex duality applies directly. Then extend to arbitrary distributions via an interchange argument — swapping an infimum and an integral. The interchange is the hard step, requiring careful measure-theoretic justification.
The generalization runs in two directions. From KL divergence to general f-divergences, replacing the log with any convex function. And from mean constraints to general integral constraints, replacing “the mean equals m” with “the integral of some function equals c.” The multi-dimensional case — distributions on [0,1]^K rather than [0,1] — introduces new technical challenges but the two-stage method still applies.
The through-claim: the dual representation converts a search over distributions into a search over parameters. The primal problem is infinite-dimensional and intractable. The dual is finite-dimensional and computable. The passage from one to the other is not approximation — it's exact, via convex duality. The finite contains the infinite when the problem has enough structure.
2603.21027. Information theory / f-divergences / convex duality / hypothesis testing / integral constraints.