Algorithmic accountability is usually framed as a transparency problem. If affected people could understand how the algorithm reached its decision — the features it weighted, the thresholds it applied — they could contest unfair outcomes. This framing has produced a decade of research on explanations: feature importances, counterfactual examples, attention visualizations.
Stewart (arXiv:2603.22716) analyzed 168 legal cases challenging algorithmic decisions and found that transparency is the wrong axis. The cases revealed a two-gate structure: first, procedural access to evidence about how the system actually works; second, substantive liability rules that determine whether the evidence matters.
The numbers are stark. Cases that lacked evidentiary access succeeded 9% of the time. Cases with evidentiary access succeeded 97% of the time in domains without liability protections. The bottleneck isn't understanding — it's access. You can explain a decision perfectly, but without the ability to test the system with modified inputs, to verify that the explanation matches the actual decision process, the explanation is unfalsifiable.
The proposed solution is counterfactual interrogation rights: the ability to submit modified inputs and observe changed outputs, without requiring disclosure of the model itself. This is a black-box audit right — testing the system's behavior without opening it. The accountability shifts from “explain how you decided” to “prove your decision would survive if the protected characteristic were different.”
The through-claim: accountability that offers explanation without evidence is procedurally decorative. The 9% success rate without access isn't a gap in understanding — it's a gap in power. Transparency without testability is a wall painted to look like a window.