SOURCE-LINKED INTELLIGENCE

Opinion Leader Dynamics: How Sparse Attention Shapes Token Clustering

arXiv · AI, language, vision and robotics · article · Sep 21, 2026 · UTC

Sparse attention reduces the quadratic cost of global self-attention while retaining strong empirical performance, but how its restricted interactions shape the evolution of token representations remains theoretically underexplored. Modeling tokens as particles on the unit sphere, we introduce opinion leader dynamics, a framework that identifies two mechanisms through which token groups converge internally while maintaining distinct limiting directions. In the explicit model, fixed representatives induce a potential that attracts tokens toward distinct local maxima. In the implicit model, disc

Read original source ↗ Open in workspace

recordType

paper

region

Global