SOURCE-LINKED INTELLIGENCE

DeltaS: Reading the Gated Linear Attention State for KV Cache Eviction in Streaming Video

arXiv · AI, language, vision and robotics · article · Sep 23, 2026 · UTC

Recent video-language models increasingly adopt hybrid architectures that interleave linear and full attention layers for efficient long-context processing. While the recurrent state of linear attention remains fixed in size, the KV cache of full attention continues to grow with the video stream, making eviction necessary under a bounded memory budget. The key challenge in streaming is that eviction must occur before the question arrives, so what to retain has to be decided without the question. Existing eviction methods derive token scores from the KV cache itself, using position, attention,

Read original source ↗ Open in workspace

recordType

paper

region

Global