SOURCE-LINKED INTELLIGENCE

Structure Aware Neural Architecture Search for Mixture of Experts

arXiv · AI, language, vision and robotics · article · Aug 30, 2026 · UTC

Neural Architecture Search (NAS) has so far rarely been applied to Mixture-of-Experts (MoE) models, and existing MoE designs leave the alignment between experts and the structure of the data to emerge on its own. We propose an architecture search framework that makes this alignment an explicit search variable: the assignment of data clusters to experts is optimised jointly with the per-expert architectures. We cast the joint problem as a cluster-aware likelihood maximisation, show that it coincides with the incomplete-data maximum likelihood of a latent-variable mixture, and solve it by a gene

Read original source ↗ Open in workspace

recordType

paper

region

Global