SOURCE-LINKED INTELLIGENCE

RouteSparse: Input-Conditional Pattern Routing for Budgeted Long-Context Prefilling

arXiv · AI, language, vision and robotics · article · Aug 29, 2026 · UTC

Dynamic sparse attention can reduce the quadratic cost of long-context prefilling without changing model weights. MInference assigns each attention head one pattern offline and estimates that pattern's sparse indices for every prompt. This design is efficient, but it assumes that a head's preferred pattern and sparsity budget remain suitable across inputs. We introduce RouteSparse, which routes each head and prompt segment among a small library of GPU-efficient sparse patterns. A low-cost probe estimates pattern utility and uncertainty; a latency-aware router then selects a pattern and budget,

Read original source ↗ Open in workspace

recordType

paper

region

Global