SOURCE-LINKED INTELLIGENCE

CateKV: On Sequential Consistency for Long-Context LLM Inference Acceleration

arXiv · AI, language, vision and robotics · article · Aug 31, 2026 · UTC

Large language models (LLMs) have demonstrated strong capabilities in handling long-context tasks, but processing such long contexts remains challenging due to the substantial memory requirements and inference latency. In this work, we discover that certain attention heads exhibit sequential consistency in their attention patterns, which can be persistently identified using a coefficient-of-variation-based algorithm. Inspired by this observation, we propose CateKV, a hybrid KV cache method that retains only critical token information for consistent heads, thereby reducing KV cache size and com

Read original source ↗ Open in workspace

recordType

paper

region

Global