SOURCE-LINKED INTELLIGENCE

Federated Multilingual Speech-LLMs: Architecture and Aggregation Strategy Benchmarking

arXiv · AI, language, vision and robotics · article · Sep 20, 2026 · UTC

We present a comprehensive benchmark of Federated Learning (FL) for multilingual Automatic Speech Recognition (ASR), evaluating four Speech-LLM architectures on the Multilingual LibriSpeech dataset. We compare FedAvg and FedProx across frozen and unfrozen encoder configurations, demonstrating that optimized learning rates are critical for performance. Specifically, independently tuning the learning rates for the speech encoder, connector, and decoder yields the lowest error rates, with full three-component adaptation (LoRA for encoder and decoder, full training for the connector) producing the

Read original source ↗ Open in workspace

recordType

paper

region

Global