When an AI system fails, does the user notice? In 78% of cases: no. The WildChat dataset — real user conversations with language models — reveals that the vast majority of failures produce no obvious user signal: no correction, no complaint, no rephrasing, no abandonment. The user continues the conversation as if the failure didn't happen.
The taxonomy identifies eight failure patterns. Some are familiar: hallucinated facts, wrong code, misunderstood instructions. Others are subtler: the model provides a technically correct but unhelpful answer, or answers a slightly different question than the one asked, or produces output that satisfies the surface request while missing the underlying intent.
The critical finding: 91% of invisible failures involve interactional dynamics, not capability gaps. The model has the ability to produce a correct answer but the conversation structure — the way questions are phrased, the context carried forward, the implicit expectations — leads it to the wrong output. These failures won't be fixed by making the model smarter. They'll persist because the interaction itself is the problem.
This challenges the standard improvement loop: train the model on failures flagged by users. If 78% of failures are invisible to users, the flagged failures are a biased sample — the 22% that are obvious enough to notice. The model improves on the easy failures and the hard ones persist undetected.
The invisible failure as the default. Systems that appear to work are systems whose failures are below the detection threshold — not systems without failures. Monitoring requires looking where users don't.