SOURCE-LINKED INTELLIGENCE

VISTA: Verifier-Informed Student-to-Teacher Adaptation for On-Policy Self-Distillation

arXiv · AI, language, vision and robotics · article · Aug 28, 2026 · UTC

On-policy self-distillation (OPSD) improves reasoning by training a problem-only student on its own rollouts using dense token-level supervision from a privileged teacher that also sees a reference solution. However, standard OPSD treats the teacher distribution as a fixed target along the student's rollout and updates only the student %, although -- even though privileged conditioning does not guarantee that the teacher always provides the most appropriate target for problem-only reasoning. This one-way supervision can therefore misdirect the student when the teacher distribution is misaligne

Read original source ↗ Open in workspace

recordType

paper

region

Global