SOURCE-LINKED INTELLIGENCE

Distill What You Trust: Reliability-Aware Multi-Teacher On-Policy Distillation

arXiv · AI, language, vision and robotics · article · Sep 20, 2026 · UTC

Multi-teacher on-policy distillation allows a student to learn from complementary specialists on its own trajectories. Domain-routed approaches, however, select one teacher per example and keep it fixed throughout the response. This design both depends on labels that mixed training corpora often lack and cannot adapt teacher selection when the expertise required changes within a trajectory. We propose \textbf{TrustMOPD}, which replaces example-level teacher selection with label-free, token-level supervision allocation. At each student-generated prefix, TrustMOPD uses each specialist's RL-induc

Read original source ↗ Open in workspace

recordType

paper

region

Global