SOURCE-LINKED INTELLIGENCE

SpecQuant: Speculative Decoding with Multi-Parent Quantization for Adaptive LLM Inference

arXiv · AI, language, vision and robotics · article · Sep 18, 2026 · UTC

Running large language models (LLMs) locally continues to be limited by restrictions of compute and memory on consumer hardware. The popular acceleration technologies, such as quantization, speculative decoding, and adaptive inferencing, offer substantial speed boosts but usually necessitate retraining, per architecture tuning, or draft models. SpecQuant is a trainingfree framework, that combines speculative decoding with multiparent quantization to perform adaptive, efficient inference of LLMs. SpecQuant derives multiple quantized variants (INT4, FP8, FP16) from a shared base model, and dynam

Read original source ↗ Open in workspace

recordType

paper

region

Global