SOURCE-LINKED INTELLIGENCE

Reading Right, Answering Wrong: How Visual Configuration Changes Affect Evidence Use in VLMs

arXiv · AI, language, vision and robotics · article · Sep 22, 2026 · UTC

Vision-language models (VLMs) have achieved strong performance on tasks such as visual question answering, yet small image resizes can turn correct answers into errors. We investigate whether changes in visual configuration, such as image tiling and token arrangement, contribute to this instability. Across seven checkpoints and four benchmarks, equally small resizes cause more correctness flips when they switch configurations. Surprisingly, in over half of these cases, models answer the question incorrectly but can still read the correct answer when told what to read. Furthermore, attention in

Read original source ↗ Open in workspace

recordType

paper

region

Global