SOURCE-LINKED INTELLIGENCE

D$^3$-MOPD: Dynamic Domain ScheDuling for Efficient Multi-Teacher Distillation

arXiv · AI, language, vision and robotics · article · Aug 25, 2026 · UTC

Multi-teacher on-policy distillation (MOPD) distills several domain-expert teachers into a single student by minimizing per-domain reverse-KL divergence on the student's own rollouts. Existing approaches typically fix the per-domain data mixture before training, overlooking the fact that different domains converge at substantially different rates: some plateau early while others continue to improve throughout the training budget. A fixed mixture therefore wastes compute on fast-converging domains and undertrains slower-converging ones. To address this, we propose D$^3$-MOPD (Dynamic Domain Sch

Read original source ↗ Open in workspace

recordType

paper

region

Global