SOURCE-LINKED INTELLIGENCE

MAPS: Memory-Aware Predictive Scheduling Framework for Large Language Model Serving

arXiv · AI, language, vision and robotics · article · Sep 14, 2026 · UTC

The surge of large language model (LLM) applications on personal devices imposes massive, bursty workloads on cloud serving infrastructure. While prefill-decode disaggregation improves throughput and scalability, memory-bound decode instances often suffer from persistent load imbalance, as output lengths are unknown when requests arrive at the cloud. To address this, we propose MAPS, a Memory-Aware Predictive Scheduling framework tailored for disaggregated LLM serving. MAPS performs device-assisted speculative output length prediction overlapped with cloud-side prefilling, incurring negligible

Read original source ↗ Open in workspace

recordType

paper

region

Global