SOURCE-LINKED INTELLIGENCE

1% of Tokens Can Be Enough: On Gradient Estimation in On-Policy Distillation

arXiv · AI, language, vision and robotics · article · Sep 21, 2026 · UTC

Sparse on-policy distillation (OPD) allocates teacher supervision to a small subset of tokens in student-generated trajectories. However, useful teacher guidance can yield a noisy update when its gradient is estimated from a sampled next token. We study this estimation problem at a fixed prefix in information geometry and propose an information-efficiency ratio (IER) based on a signal-to-noise decomposition. IER characterizes relative gradient estimation error under an optimal scalar baseline. A candidate-set approximation enables token selection based on IER and its combination with existing

Read original source ↗ Open in workspace

recordType

paper

region

Global