SOURCE-LINKED INTELLIGENCE

Disaggregated Quantization: Specializing LLM Prefill and Decode

arXiv · AI, language, vision and robotics · article · Sep 22, 2026 · UTC

Prefill and decode reward different approaches to quantization: low-precision arithmetic accelerates prompt processing, while compact weights reduce memory traffic during generation. We propose "disaggregated quantization" (DQ), which specializes computation formats, weights and storage placement to both of these phases. On Qwen 3 and Gemma 3, removing activation quantization specifically on decode improves accuracy on decode-heavy tasks without increasing inference cost. Training separate compute-native prefill weights accelerates prompt processing relative to weight-only inference while matc

Read original source ↗ Open in workspace

recordType

paper

region

Global