SOURCE-LINKED INTELLIGENCE

AsymSpec: Context-Asymmetric Speculative Decoding for Agentic LLMs

arXiv · AI, language, vision and robotics · article · Aug 26, 2026 · UTC

Agentic LLM pipelines face escalating inference costs as context accumulates across retrieval, tool use, and multi-turn interactions. To control latency, deployments routinely compress inputs, but this degrades task accuracy. Speculative decoding (SD) accelerates generation losslessly, yet it assumes the drafter and verifier share an identical context, preventing SD from resolving the accuracy-overhead trade-off. We propose AsymSpec, an asymmetric speculative decoding framework that breaks this symmetry: a lightweight drafter reads the full input while the large verifier operates on the compre

Read original source ↗ Open in workspace

recordType

paper

region

Global