SOURCE-LINKED INTELLIGENCE

Retrieval Geometry Shapes Cache-Based Clip Adaptation

arXiv · AI, language, vision and robotics · article · Sep 20, 2026 · UTC

Cache-based test-time adaptation improves CLIP predictions by storing and retrieving examples from the target stream while keeping the model frozen. However, existing methods largely treat the feature space used for image-image retrieval as fixed, leaving open how much adaptation depends on the retrieval space itself. We study this question by fixing the memory and changing only the retrieval encoder, finding that the same memory can yield very different gains: across sixteen retrieval spaces, ImageNet-A cache gain ranges from at most +0.44 points for CLIP and MAE to +19.7 +/- 0.4 for DINOv2-L

Read original source ↗ Open in workspace

recordType

paper

region

Global