In a Mixture-of-Experts transformer, each token is routed to a subset of available experts. The routing decision is supposed to be infrastructure — a dispatch mechanism that balances computational load across hardware. The standard design pressure is utilitarian: distribute tokens evenly so no expert is overloaded.
Avinash (arXiv:2603.11114) shows that the routing is not merely infrastructure. It is signal. When prompts from the same task category are fed through a sparse MoE model, their routing patterns — which experts activate, how strongly — are highly similar (cosine similarity 0.84). Prompts from different categories produce different routing patterns (0.62). A logistic regression trained on routing vectors alone achieves 92.5% task classification accuracy. The routing decision contains enough information to identify the task without examining the computation the experts perform.
The structural point is that the dispatch became a representation without being designed as one. The router was trained to distribute tokens efficiently. But efficiency, in a model trained on diverse tasks, means sending similar computations to the same experts. The experts specialize not because they were told to, but because specialization is efficient. And once they specialize, the routing pattern — the choice of which specialist to consult — encodes what kind of problem is being solved.
This is a general pattern: any system that routes signals to specialized processors will develop routing patterns that are readable. The routing is doing double duty — both dispatching the work and encoding what the work is. Reading the route tells you the task because the route IS the task classification, compressed into an activation pattern. The architecture that was designed for efficiency accidentally produced interpretability.