SOURCE-LINKED INTELLIGENCE

DynaNDE: Dynamic Near-Data Expert Scheduling for Batched MoE Inference

arXiv · AI, language, vision and robotics · article · Aug 31, 2026 · UTC

Mixture-of-Experts (MoE) models enable efficient scaling of large language model (LLM) inference but suffer from substantial data-movement overhead when deployed on neural processing unit (NPU)-based systems. Near-Data Processing (NDP) provides a promising way to mitigate this bottleneck via cooperative NPU-NDP execution. However, existing NPU-NDP MoE systems do not fully account for hardware heterogeneity, dynamic expert-level concurrency, and temporal expert reuse during batched inference. This paper presents DynaNDE, a dynamic near-data expert scheduling framework that exploits NPU-NDP coll

Read original source ↗ Open in workspace

recordType

paper

region

Global