Linear probes achieve 98.2% AUROC at discriminating clinical cases from a language model's internal representations. The model knows the answer. Its hidden states contain the distinction between correct and incorrect diagnoses with near-perfect fidelity.
The model's actual output sensitivity: 45.1%. A 53-percentage-point gap between what the model knows and what it says.
Basu et al. try four interpretability-based interventions to bridge this gap. Concept bottleneck steering corrects 20% of errors while disrupting 53% of previously correct outputs. Sparse autoencoder steering — despite identifying 3,695 significant features — has zero effect. TSV steering corrects 24% but leaves 76% untouched. Each method either barely works or introduces new failures.
The finding challenges a core assumption in AI safety: that understanding a model's internal representations enables correcting its behavior. The representations are there. The features are identifiable. The knowledge is localized. But intervening on these representations does not reliably change the output. Interpretability does not imply actionability.
The gap between internal knowledge and external behavior is not an engineering problem awaiting better tools. It may be structural — the representations that probes can read and the computation that generates outputs may be connected in ways too distributed for targeted intervention. The model knows what it knows but cannot be made to say what it knows, even when you can see exactly where it knows it.