SOURCE-LINKED INTELLIGENCE

What, Where, and How: Probing Spatiotemporal Representations in Video Foundation Models

arXiv · AI, language, vision and robotics · article · Sep 1, 2026 · UTC

Self-supervised video foundation models learn rich spatiotemporal representations, yet it remains unclear what visual concepts these representations encode, where they emerge across transformer layers, and how they are geometrically organized. In this work, we tackle these three questions through a systematic layer-wise analysis of V-JEPA 2 and VideoMAE-v2. We leverage lightweight probes trained to discover three temporally grounded properties: (i) camera motion understanding, (ii) intuitive physics, and (iii) anomaly detection. Both models encode camera motion, with best results ($>90$ ROC AU

Read original source ↗ Open in workspace

recordType

paper

region

Global