SOURCE-LINKED INTELLIGENCE

Block-Sparse Attention with Semantic-Geometric Decoupled Routing

arXiv · AI, language, vision and robotics · article · Sep 19, 2026 · UTC

Long-context inference has become a defining capability of large language models, but exact dense attention remains costly due to its quadratic scaling with sequence length. Block-sparse attention offers a hardware-friendly alternative by routing each query block to a small set of relevant key blocks, yet accurate training-free block routing remains difficult. Existing routers often pool post-RoPE token representations, which entangles semantic aggregation with RoPE-induced geometry and attenuates local positional cues through high-frequency phase cancellation. To resolve this mismatch, we pro

Read original source ↗ Open in workspace

recordType

paper

region

Global