SOURCE-LINKED INTELLIGENCE

RISE: Recursive Improvement via Self-Extrapolating Policy Distillation

arXiv · AI, language, vision and robotics · article · Sep 4, 2026 · UTC

On-policy distillation (OPD) provides dense, per-token supervision for language model post-training, but its effectiveness is bottlenecked by teacher quality: external teachers suffer from distribution mismatch, while self-distillation with privileged conditioning is limited by in-context learning capacity. We propose \textbf{RISE} (\textbf{R}ecursive \textbf{I}mprovement via \textbf{S}elf-\textbf{E}xtrapolating Policy Distillation), which constructs a synthetic teacher directly from the model's own RLVR training trajectory. By extrapolating the displacement between the current checkpoint and

Read original source ↗ Open in workspace

recordType

paper

region

Global