SOURCE-LINKED INTELLIGENCE

Video-STLayout Pre-training

arXiv · AI, language, vision and robotics · article · Sep 21, 2026 · UTC

In recent years, pre-training has become fundamental to learning effective video representations, enabling strong transfer to downstream tasks. A popular framework in pre-training involves aligning features of a video encoder with that of another modality, for example, language or audio. We introduce Video-STLayout pre-training, a novel strategy for obtaining rich video representations informed by spatio-temporal layout of object bounding boxes. Object layouts can easily be obtained by applying an off-the-shelf object detector on the video frames. Our method uses a contrastive loss to align vi

Read original source ↗ Open in workspace

recordType

paper

region

Global