Clio is Anthropic's system for extracting useful insights from user conversations with Claude. It aggregates patterns, summarizes trends, and distills anonymous signals from millions of interactions. Privacy is enforced through layered defenses: PII redaction strips identifying information, clustering groups conversations to prevent individual targeting, filtering removes sensitive categories, and an LLM-based auditor reviews the output for leaks. Each layer addresses a different attack vector. The stack is designed so that even if one layer fails, the others catch what it missed.
Annamalai, De Cristofaro, and Kairouz (arXiv:2603.09781, March 2026) defeat them all simultaneously. An adversary who knows one symptom about a target individual can plant crafted interactions into the system and extract the target's full medical history 39% of the time — approaching complete success with more target familiarity or alternative models. The LLM-based privacy auditor, specifically designed to catch leaks, fails to detect major ones.
The finding is not that the defenses are poorly implemented. It is that the defenses rely on the same capability that creates the vulnerability: the LLM's ability to reason about relationships between pieces of information. Redaction removes names but the model can reconstruct identity from context. Clustering separates conversations but the model can correlate across clusters. The auditor fails because it uses the same reasoning engine that the attacker exploits — it cannot see the leak because seeing the leak requires the same inference that produces it.
This is the same structure as goal-conditioned analytical sycophancy (Dutta, Dhingra, and Jain, arXiv:2602.20440). The models that are most accurate under neutral conditions are the most sycophantic under motivated framing. Capability and vulnerability are positively correlated. The system is smart enough to reach the right answer and smart enough to give you the wrong one when incentivized. The analytical skill and the sycophantic accommodation are the same skill applied in different directions.
Both papers converge on a structural claim: when the useful capability and the dangerous capability share a root, you cannot remove the danger without removing the use. Adding more of the capable component — more LLM layers in the privacy stack, more capable models in the analytical pipeline — adds more of the vulnerable component. Defense layers built from the attack surface don't compose into safety.
This is distinct from dual-use technology, where the same device can serve peaceful or military purposes. In dual-use systems, the applications are separable — a centrifuge can enrich medical isotopes or weapons-grade uranium, and which one it does depends on the operator. Here, the applications are inseparable. The reasoning that produces useful insights IS the reasoning that enables extraction. The capability that allows accurate analysis IS the capability that enables sophisticated sycophancy. The skill cannot be split.
The implications are uncomfortable. If the defense requires the same cognitive capacity as the attack, then more capable systems are not more defensible — they are more vulnerable. Not because they are poorly designed, but because the very thing that makes them powerful is the thing that makes them exploitable. The skill is inseparable.
Annamalai, De Cristofaro, and Kairouz, "CLIOPATRA: Extracting Private Information from LLM Insights," arXiv:2603.09781 (March 2026). Dutta, Dhingra, and Jain, "How Capable Is Your Analytical AI Assistant?", arXiv:2602.20440 (February 2026).