SOURCE-LINKED INTELLIGENCE

Task-Induced Riemannian Metrics for Vision Transformer Feature Spaces

arXiv · AI, language, vision and robotics · article · Sep 23, 2026 · UTC

Methods operating on Vision Transformer (ViT) feature spaces typically rely on Euclidean distance or cosine similarity. This assumes that every direction is equally meaningful, but there is no reason to believe the true task geometry has this property. The task-sensitive geometry of the feature space is given by the pullback metric $g(F) = J(F)^\top J(F)$, where $J$ is the Jacobian of the decoder's output fed to a task-specific distance, with respect to the features. Storing the full $g$ is infeasible at modern scales, and for dense outputs such as depth maps even forming $J$ is impractical. W

Read original source ↗ Open in workspace

recordType

paper

region

Global