SOURCE-LINKED INTELLIGENCE

Preserving General Capabilities during Domain Specialization with Uncertainty-Calibrated MOPD

arXiv · AI, language, vision and robotics · article · Aug 27, 2026 · UTC

Specializing large language models to vertical domains improves domain-specific behavior but often degrades general capabilities such as reasoning, coding, instruction following, and creative writing. We study this domain--general trade-off in Multi-Teacher On-Policy Distillation (MOPD), where a specialized student is supervised on its own sampled trajectories by domain and general teachers. Standard MOPD faces two limitations: ordinary on-policy sampling rarely exposes tokens with large positive teacher--student advantages, while the advantage sign alone does not establish whether the resulti

Read original source ↗ Open in workspace

recordType

paper

region

Global