SOURCE-LINKED INTELLIGENCE

DASC: Decay-Aware State Compression for Hybrid Linear-Attention Serving

arXiv · AI, language, vision and robotics · article · Aug 31, 2026 · UTC

Hybrid linear-attention architectures have recently scaled to large open-weight models, offering quality competitive with full attention while substantially reducing key/value (KV) cache growth. However, their in-place recurrent-state updates complicate cache management: prefix reuse requires state checkpoints alongside full-attention KV, while storing state checkpoints in full increases memory pressure, leading to more evictions and repeated prefill. By analyzing the decay structure of Gated DeltaNet (GDN) and Kimi Delta Attention (KDA), we find that different heads and channels retain prefix

Read original source ↗ Open in workspace

recordType

paper

region

Global