SOURCE-LINKED INTELLIGENCE

Low resource cross-modal alignment using HGNN to enhance speech representation

arXiv · AI, language, vision and robotics · article · Sep 19, 2026 · UTC

Speech-text space alignment is a multimodal representation learning method consisting to map different speech and text into a shared representation space, leading to enrichment of the representation of each modality. Proposed architectures, such as SAMU-XLSR, typically follow a student/teacher framework, with the goal of fine-tuning an audio encoder to produce representations that closely match those of the text. In this way a speech representation is semantically enriched. However, such systems generally require large amounts of training data and considerable computational resource, making th

Read original source ↗ Open in workspace

recordType

paper

region

Global