SOURCE-LINKED INTELLIGENCE

Meta-Learning Where to Allocate Experts: Task-Conditioned Layer-Wise Compression for MoEs

arXiv · AI, language, vision and robotics · article · Aug 27, 2026 · UTC

Mixture-of-Experts (MoE) models route each token to a subset of expert networks, increasing capacity while keeping per-token computation sparse. In many deployed MoEs, the number of active experts is fixed across layers and tasks, although layer roles and expert redundancy vary with depth and demand varies with difficulty. Existing approaches address only part of this setting: layer-wise allocations are usually determined offline and reused for all tasks, while token-level methods vary expert activation using local routing signals without task-level context. We propose MetaNet, a support-set c

Read original source ↗ Open in workspace

recordType

paper

region

Global