LLM agents are being deployed to manage wireless networks autonomously — configuring resources, diagnosing problems, making real-time decisions. But existing benchmarks evaluate these agents with exact-match metrics that miss the most dangerous failure mode: catastrophic errors where the answer is not just wrong but harmful.
The authors (arXiv:2603.21251) introduce WirelessBench, organized into three tiers of increasing complexity: domain knowledge reasoning, intent-driven resource allocation, and proactive multi-step decisions under mobility. The key design element is tolerance-aware scoring with catastrophic-error detection. A base station configured with 10× too much power isn't just inaccurate — it causes interference across the network.
Results: direct prompting achieves 68% accuracy, tool-integrated agents reach 84.64%. But 23% of errors across all agents are catastrophic — invisible to standard accuracy metrics.
The through-claim: the error distribution matters more than the error rate. An agent that's wrong 20% of the time with small errors is safer than one that's wrong 15% of the time with catastrophic errors in the 15%. Standard benchmarks, by treating all errors equally, select for agents that minimize average error rather than maximum harm. The tolerance-aware approach inverts the priority: worst-case matters more than average-case in infrastructure management.