SOURCE-LINKED INTELLIGENCE

SMART: MLLM-guided Temporal Alignment for Unifying Sign Language Recognition and Spotting

arXiv · AI, language, vision and robotics · article · Aug 26, 2026 · UTC

Continuous sign language recognition (CSLR) aims to recognize gloss sequences from unsegmented sign videos under weak sequence-level supervision. However, existing methods rely on sentence-level gloss annotations, providing limited temporal and semantic guidance for fine-grained representation learning. Conventional video-text alignment also requires large batch sizes, making it inefficient for memory-intensive sign language video training. In this work, we propose SMART, an MLLM-guided temporal alignment framework for joint sign recognition and spotting. SMART uses MLLMgenerated motion descri

Read original source ↗ Open in workspace

recordType

paper

region

Global