SOURCE-LINKED INTELLIGENCE

VCMM: Variance-Calibrated Momentum for Multimodal Learning

arXiv · AI, language, vision and robotics · article · Sep 23, 2026 · UTC

Multimodal joint training often suffers from modality imbalance, where a dominant modality suppresses the optimization of others. Existing methods mainly balance modality learning by modulating gradient magnitudes or directions, modifying optimization objectives, or adjusting training strategies, with most interventions focusing on the current update. However, when combined with widely used momentum-based optimizers, the update also incorporates accumulated information from previous gradients, which is not explicitly addressed by current-step modulation alone. To address this issue, we propose

Read original source ↗ Open in workspace

recordType

paper

region

Global