SOURCE-LINKED INTELLIGENCE

Beyond Mean Foils: Auditing Worst-Foil Specificity in Frozen CLIP Region Explanations

arXiv · AI, language, vision and robotics · article · Sep 23, 2026 · UTC

A region can overlap a target object yet contribute more to another class. We test regions selected by Cluster-based Concept Importance (CCI) in frozen CLIP. Across COCO and VOC with two checkpoints, 41.08-64.78% of regions that pass overlap and mean-contrast checks fail against the strongest competing class. Removing competitors annotated in the image leaves 39.69-63.64% failing. We then test all eight candidate regions per image. An alternative passes the test for 6.25-7.84% of failures on COCO and 27.40-31.15% on VOC. Requiring it to preserve the original target-score drop within $ε= 0.02$

Read original source ↗ Open in workspace

recordType

paper

region

Global