SOURCE-LINKED INTELLIGENCE

FedSubMuon: Communication-Efficient Federated LLM Fine-Tuning via Structured Subspace Muon

arXiv · AI, language, vision and robotics · article · Sep 5, 2026 · UTC

Federated fine-tuning adapts large language models (LLMs) to decentralized client data, but its scalability in cross-device training is often limited by the high communication cost. Muon is an optimizer that improves optimization performance by orthogonalizing momentum for matrix-valued parameters. Existing federated Muon methods demonstrate the benefit of matrix-aware optimization in federated learning, but still require transmitting full layer-size updates and optimizer state. A natural way to reduce communication is to directly apply Muon to LoRA factors, but this changes the optimized obje

Read original source ↗ Open in workspace

recordType

paper

region

Global