SOURCE-LINKED INTELLIGENCE

HySparse2: Hybrid Sparse Attention with Two-Level KV Sharing

arXiv · AI, language, vision and robotics · article · Sep 22, 2026 · UTC

Long-horizon and multi-turn agents typically generate short actions and process long observations from tools and environments. This growing context demands efficient prefill, compact KV-cache storage, and accurate long-context retrieval. To meet these demands, we introduce HySparse2, a hybrid sparse attention architecture with two-level KV sharing. At the outer level, KV Bridging adopts a YOCO-style self-decoder and cross-decoder structure, but bridges only full-attention layers. The self-decoder uses hybrid sliding-window attention (SWA), while the cross-decoder uses hybrid sparse attention.

Read original source ↗ Open in workspace

recordType

paper

region

Global