SOURCE-LINKED INTELLIGENCE

GrowPage: On-Demand KV Budgeting for Efficient LLM Reasoning Serving

arXiv · AI, language, vision and robotics · article · Sep 3, 2026 · UTC

Long-output reasoning has made the key--value (KV) cache a critical memory bottleneck for efficient LLM serving. Existing KV compression methods usually rely on a predefined per-request budget and adjust only which KV states are retained, leaving the total capacity fixed throughout decoding. However, reasoning workloads exhibit substantial demand variation: different requests require different KV capacities, and the attention demand of an individual request evolves during generation. We introduce \textbf{GrowPage}, an on-demand KV budgeting framework that treats KV capacity as a runtime resour

Read original source ↗ Open in workspace

recordType

paper

region

Global