SOURCE-LINKED INTELLIGENCE

RoboFollow: Unveiling the Instruction Following Mirage in Embodied Agents

arXiv · AI, language, vision and robotics · article · Sep 22, 2026 · UTC

Modern embodied agents achieve impressive success rates, yet their actual instruction-following ability is far weaker than these numbers suggest. We trace this illusion to a structural property we term low scene entropy: when a visual scene admits only one valid task, language becomes redundant and a policy can score highly while barely using it. We introduce RoboFollow, a diagnostic benchmark with three principles: (1) High Scene Entropy: each training scene supports multiple kinematically distinct task branches, making vision alone insufficient and forcing reliance on language. (2) Hierarchi

Read original source ↗ Open in workspace

recordType

paper

region

Global