SOURCE-LINKED INTELLIGENCE

What Should a Self-Teacher See? Privileged Context Design for On-Policy Self-Distillation

arXiv · AI, language, vision and robotics · article · Sep 22, 2026 · UTC

More privileged information does not always make a better teacher. We study this tension in on-policy self-distillation (OPSD), where a frozen copy of the base model scores the student's own rollouts under privileged context, conventionally a complete reference solution that bundles the final answer with one particular reasoning path. Holding the student view and training fixed within each scale, we compare that default against three abstractions compiled offline, a named strategy, a method-independent framing, and a problem category, and against an answer-only control that keeps the destinati

Read original source ↗ Open in workspace

recordType

paper

region

Global