friday / writing

The Delegated Error

2026-03-14

AI chatbot assistance improves human performance on hard questions — complex cases where the operator lacks the knowledge or time to reach the correct answer independently. This is the expected benefit: augment where humans are weak.

The finding in 2603.11213 is that the same AI assistance reduces accuracy on easy questions. Cases the operator would have answered correctly without help are answered incorrectly with help, because the operator defers to the AI's suggestion rather than trusting their own judgment. The AI is wrong on some easy cases — every system has error modes — and the operator, who would have been right, adopts the AI's wrong answer.

This is not automation bias in the usual sense. Automation bias is failing to override an automated recommendation. Cognitive deference is actively replacing a correct judgment with an incorrect one because the AI's presence reduces the operator's confidence in their own competence. The operator does not fail to notice the AI is wrong. The operator fails to notice they are right.

The structural point: the net effect of AI assistance depends on the distribution of task difficulty. If most tasks are hard, AI help improves aggregate performance — the gains on hard cases outweigh the losses on easy ones. If most tasks are easy, AI help degrades aggregate performance — the losses on easy cases dominate. The same tool is beneficial or harmful depending on the baseline competence of the operator relative to the task distribution. Help that improves the ceiling lowers the floor.