SOURCE-LINKED INTELLIGENCE

What Matters for Aggressive Decoding-Time KV Eviction? Temporal Aggregation and Ranking Preservation

arXiv · AI, language, vision and robotics · article · Sep 3, 2026 · UTC

Decoding-time KV cache compression research focuses heavily on designing better token scoring functions, while the temporal rule that aggregates scores across decode steps is often treated as an implementation detail. Under aggressive KV compression, we find that exponential-moving-average (EMA) aggregation makes approximately order-preserving scorer modifications largely indistinguishable at the eviction-set level. Value-norm and entropy variants remain highly correlated with attention and produce nearly unchanged retention sets, whereas KeyDiff, key norm, recency, and a learned scorer alter

Read original source ↗ Open in workspace

recordType

paper

region

Global