SOURCE-LINKED INTELLIGENCE

EStream: Fast and Memory-Efficient MoE Prefill through Expert Virtualization on Mobile NPUs

arXiv · AI, language, vision and robotics · article · Sep 6, 2026 · UTC

Mobile vendors and application developers increasingly deploy LLMs on smartphones for diverse prefill-only services. Yet current systems rely mainly on dense models whose regular computation maps efficiently to mobile NPUs, leaving more capable MoEs underused. MoE prefill does not fit mobile NPUs: NPU graphs are fixed at compile time, yet MoE picks experts at runtime; and one request touches most experts, more than a phone can hold in memory. We present EStream, which resolves both by separating what the NPU must fix from what MoE decides at runtime. A single compiled expert graph serves every

Read original source ↗ Open in workspace

recordType

paper

region

Global