SOURCE-LINKED INTELLIGENCE

From Concentration to Differentiation and Back: Routing Effective Rank in MoE Reasoning Cohorts

arXiv · AI, language, vision and robotics · article · Sep 6, 2026 · UTC

Test-time scaling produces cohorts of reasoning rollouts, yet there is no standard label-free account of how their internal computation reorganizes as inference unfolds. We introduce routing effective rank deff, the entropy-effective dimensionality of a cross-rollout graph built from MoE expert-routing similarity. Across ten MoE configurations and five math/science benchmarks, deff exhibits a reproducible low-high-low trajectory, with a prominent interior maximum in 98.5% of 3,105 model-question cohorts: routing similarity is concentrated early, maximally differentiated at intermediate budgets

Read original source ↗ Open in workspace

recordType

paper

region

Global