SOURCE-LINKED INTELLIGENCE

XYEval: Agents say yes to bad advice

arXiv · AI, language, vision and robotics · article · Sep 20, 2026 · UTC

Effective communication between users and AI agents is essential for human-AI collaboration. The XY problem is a well-known communication pitfall where a person asks about their attempted solution rather than their actual problem. We extend prior sycophancy evaluation to the XY problem in agentic settings, evaluating whether agents can resist plausible but misleading suggestions from users and communicate their reasoning. We introduce XYEval, a meta-evaluation framework that can transform an existing benchmark into an XY problem evaluation. We evaluate five models across six diverse benchmark

Read original source ↗ Open in workspace

recordType

paper

region

Global