In multimodal learning, the stronger modality dominates. Vision overpowers audio. Text overpowers everything. The standard fix: reweight the modalities, penalizing the dominant one to force balance.
Xiong et al. argue the fix is wrong because the framing is wrong. The problem is not that strong modalities dominate. The problem is that each modality has a finite information budget — a maximum amount of task-relevant information it can contribute — and the training process does not respect these budgets.
IIBalance estimates each modality's information budget through a task-grounded estimator. Then it applies correction only where needed: a prototype-based alignment mechanism that intervenes when weaker modalities deviate from their budgeted potential, and stays silent when they are on track. At inference, a probabilistic gating module generates calibrated fusion weights — not fixed reweighting, but dynamic allocation based on what each modality actually contributes to each input.
The structural insight is in the asymmetry of correction. Forced balance treats all modalities as if they should contribute equally. But modalities are not equal — they contain different amounts of task-relevant information, and the optimal fusion weights depend on the input. A visual scene with rich texture and ambiguous audio should weight vision heavily. The same scene with clear speech and occluded objects should weight audio heavily. Forced balance prevents both.
The budget framing transforms the problem from “how to suppress the strong modality” to “how to help each modality reach its potential.” The strong modality is not the enemy. The weak modality underperforming its budget is the problem. Correct the underperformer instead of handicapping the leader.