SOURCE-LINKED INTELLIGENCE

ResiSpec: Enhancing Multi-Candidate Speculative Sampling via Residual Distribution Shaping

arXiv · AI, language, vision and robotics · article · Aug 25, 2026 · UTC

The efficiency of Large Language Model (LLM) serving is fundamentally limited by the sequential nature of autoregressive decoding. Speculative Decoding (SD) mitigates this by using a lightweight draft model to speculate future tokens, which are then validated by the LLM in a single parallel forward pass. To further boost efficiency, multi-candidate schemes propose diverse candidate sets to increase the likelihood of token acceptance. However, we show that these schemes are bottlenecked by Residual Drift: a phenomenon where the rejection of initial candidates causes the residual target distribu

Read original source ↗ Open in workspace

recordType

paper

region

Global