SOURCE-LINKED INTELLIGENCE

SignGPT: Toward LLM-Mediated Sign Language Interaction through Gloss-Free Translation and Generation

arXiv · AI, language, vision and robotics · article · Sep 18, 2026 · UTC

Large language models (LLMs) provide limited support for sign language interaction. Unifying sign language translation (SLT) and generation (SLG) to enable sign language as both input and output can reduce switching between separate models during sign-text interaction. We present SignGPT, a unified, pose-based framework for gloss-free SLT and SLG. SignGPT integrates part-aware hierarchical representations of body, hand, and facial motion into a shared language model and employs asymmetric multi-token prediction and progressive training for bidirectional modeling. We evaluate SignGPT on How2Sig

Read original source ↗ Open in workspace

recordType

paper

region

Global