SOURCE-LINKED INTELLIGENCE

SGD-KV: Summarization Guided KV Cache Compression

arXiv · AI, language, vision and robotics · article · Sep 3, 2026 · UTC

Large language models (LLMs) face severe memory bottlenecks in long-context inference due to the linearly growing size of key-value (KV) caches. Existing KV cache compression techniques typically rely on simple heuristics, overlooking the distinct functional roles of different attention heads. We present SGD-KV (Summarization-Guided KV Cache Compression), a head-aware framework that leverages a novel chunk-summarization diagnostic task to systematically identify and prioritize attention heads specialized in hierarchical information aggregation. Experiments on Qwen2.5-7B-1M and Qwen3-32B across

Read original source ↗ Open in workspace

recordType

paper

region

Global