SOURCE-LINKED INTELLIGENCE

CoRe-MoE: Compact Reusable MoE for Continual Multimodal Instruction Tuning

arXiv · AI, language, vision and robotics · article · Aug 28, 2026 · UTC

Continual multimodal instruction tuning requires multimodal large language models to acquire new task abilities sequentially while preserving previously learned knowledge. LoRA-MoE provides a promising solution by introducing expert-based capacity, but repeatedly learning and maintaining full LoRA experts leads to substantial parameter overhead. This raises a natural question: is full expert expansion necessary for every new task? To answer it, we analyze the SVD of task-specific LoRA updates and observe substantial overlap in their input- and output-side LoRA direction subspaces, with task-sp

Read original source ↗ Open in workspace

recordType

paper

region

Global