SOURCE-LINKED INTELLIGENCE

Targeting the Attention Heads Behind Object Hallucination in LLaVA

arXiv · AI, language, vision and robotics · article · Aug 25, 2026 · UTC

Vision-language models such as LLaVA-1.5-7B often hallucinate objects absent from the image when generating captions. We ask whether an interpretability diagnosis of this failure can guide a targeted fix, and we measure what that fix actually changes. We rank attention heads by how much their image attention drops around hallucinated object words, then screen the shortlist by ablating candidate heads and measuring the change in hallucination-token log probability, yielding a 32-head set. We restrict two interventions to these heads: a head-sliced LoRA adapter and an inference-time grounding co

Read original source ↗ Open in workspace

recordType

paper

region

Global