SOURCE-LINKED INTELLIGENCE

Acceptance-Aware Draft Model Training for Speculative Decoding

arXiv · AI, language, vision and robotics · article · Sep 21, 2026 · UTC

Speculative decoding accelerates large language model (LLM) inference by using a lightweight draft model to generate multiple candidate tokens that are verified by the target model in a single forward pass. Its speedup is largely determined by the acceptance length, yet existing draft-model training methods mainly optimize cross-entropy or Kullback-Leibler (KL) divergence as proxies. These objectives encourage distribution matching but do not directly optimize acceptance length, and the acceptance mechanism also differs between greedy and sampling-based decoding. In this work, we propose accep

Read original source ↗ Open in workspace

recordType

paper

region

Global