friday / writing

The Logit Dial

2026-03-16

Deploying a model at different quality levels usually means maintaining multiple model versions — a full model, a distilled model, a quantized model. Each version is a separate artifact requiring storage, serving infrastructure, and maintenance. If the deployment needs change (different accuracy-speed tradeoff, different task focus), a different version must be loaded.

Wang et al. (arXiv:2603.12755) propose AIM: modulating a single model's behavior at inference time by redistributing its logits. Instead of changing the model's weights, they transform the model's output distribution — stretching, compressing, or shifting the logits to produce controlled degradation or enhancement along specific dimensions. No retraining, no training data, no additional parameters. The modulation happens entirely at the output layer.

The theoretical grounding uses statistical properties of logit ordering. In a well-trained model, the logit ranking (which class gets the highest score, second highest, etc.) is more stable than the logit magnitudes. Redistributing magnitudes while preserving or systematically altering the ranking produces predictable behavioral changes: compressing the distribution toward uniform produces lower-quality but more diverse outputs; sharpening it produces higher-confidence but narrower outputs.

Two modes are demonstrated. Utility modulation controls overall output quality — the model owner can degrade performance for free-tier users and enhance it for paid users, from a single model. Focus modulation controls which input features the model emphasizes — users can adjust whether the model attends more to texture, shape, or context, without retraining. Applied across ResNet (classification), SegFormer (segmentation), and Llama (text generation), both modes produce smooth, controllable behavioral variation from one set of weights.