SOURCE-LINKED INTELLIGENCE

Temperature-Adaptive Transformed Teacher Matching

arXiv · AI, language, vision and robotics · article · Aug 29, 2026 · UTC

Temperature scaling is a core component of knowledge distillation, yet its role and effect are still not fully understood. Transformed Teacher Matching (TTM) clarifies the role of temperature scaling by applying it only to the teacher distribution and interpreting the resulting objective as standard distillation with an implicit Rényi entropy regularization on the student. However, TTM still relies on a fixed temperature and does not specify how the teacher-side temperature should be adapted for individual samples. In this paper, we introduce a sample-wise inverse-temperature update for TTM by

Read original source ↗ Open in workspace

recordType

paper

region

Global