SOURCE-LINKED INTELLIGENCE

Part Grounding, Not Action Knowledge: Locating the Bottleneck in VLM Affordance Prediction

arXiv · AI, language, vision and robotics · article · Sep 1, 2026 · UTC

Benchmarks agree that vision-language models reason poorly about low-level manipulation, but an aggregate accuracy score does not say which step fails. We separate two steps that affordance questions conflate: identifying which part of an object to act on, and knowing what action that part requires. Across 19 articulated objects we asked eight models, spanning three developers, what motion a robot should apply. Under an open prompt, push was produced once in 64 evaluations where it was correct, despite being correct for 8 of 19 objects and appearing in the offered label set every time. Inspect

Read original source ↗ Open in workspace

recordType

paper

region

Global