SOURCE-LINKED INTELLIGENCE

When Muon Meets Task Interference: A Spectral Perspective on Continual Learning and Model Merging

arXiv · AI, language, vision and robotics · article · Aug 27, 2026 · UTC

Continual learning (CL) and model merging (MM) both aim to obtain a single model that performs well across multiple tasks, challenged respectively by catastrophic forgetting and weight-disentanglement error. In the literature, these difficulties are merely treated separately and mitigated through a variety of solutions, while the geometry induced by the base optimizer is treated as an implementation detail. In this work, we show that the two difficulties are in fact two instances of the same phenomenon: a parameter update useful for one task shifts the model's outputs on another. We formalize

Read original source ↗ Open in workspace

recordType

paper

region

Global