SOURCE-LINKED INTELLIGENCE

Trust the Mass: Forced Weights in KV-Cache Eviction

arXiv · AI, language, vision and robotics · article · Aug 25, 2026 · UTC

Every deployed sparse-attention or KV-cache-eviction rule keeps a subset of the keys, discards the rest, and renormalizes the attention weights over the kept set. Enumerating the exact best subset under that constraint on $168{,}192$ attention rows from five models shows that keeping the largest weights is already near-optimal, since the best subset closes only a median $2$ to $5\%$ of the remaining gap to full attention. If selection closes this little, published margins between eviction methods must come from elsewhere, so we measure the bytes each method holds. In the shared evaluation pipe

Read original source ↗ Open in workspace

recordType

paper

region

Global