SOURCE-LINKED INTELLIGENCE

Evidence for Shared Routing Geometry and Dynamics in Sparse Mixture-of-Experts

arXiv · AI, language, vision and robotics · article · Sep 2, 2026 · UTC

Sparse mixture-of-experts (MoE) models use an independently parameterized router at each sparse layer to select experts for every token. Prior work has shown that routing decisions across depth can often be predicted from earlier routing signals, suggesting that routing is not fully independent across layers. However, the structure behind this predictability remains unclear. In this work, we provide evidence that routing-relevant states across layers share a common geometric structure that is obscured by layer-specific coordinate systems. We isolate the control subspace of each router and alig

Read original source ↗ Open in workspace

recordType

paper

region

Global