SOURCE-LINKED INTELLIGENCE

Adapting Tree-Structured Speculative Decoding to DeepSeek-V4 for Efficient Inference

arXiv · AI, language, vision and robotics · article · Sep 21, 2026 · UTC

Repeated execution of the target model during autoregressive decoding is a major source of LLM inference latency. Unlike linear speculation, which follows a single candidate chain, tree-structured speculation retains multiple branches from shared prefixes; under the same budget, this broader coverage can improve acceptance and efficiency. Adapting it to DeepSeek-V4 is nontrivial: its CSA/HCA online compressed attention concentrates the difficulty on the target-verify side, where branches diverging from a shared prefix compress into different states, breaking cross-branch state consistency. We

Read original source ↗ Open in workspace

recordType

paper

region

Global