SOURCE-LINKED INTELLIGENCE

DAMP: Decay-Aware Mixed-Precision Recurrent-State Quantization

arXiv · AI, language, vision and robotics · article · Aug 27, 2026 · UTC

Softmax attention stores key and value vectors for every preceding token, causing inference memory to grow with sequence length. Recent language models incorporating Gated DeltaNet (GDN) or Kimi Delta Attention (KDA) reduce this cost by replacing the KV cache in most layers with fixed-size recurrent states. However, these recurrent states are commonly stored in FP32 and consume substantial GPU memory; their updates are memory-bandwidth bound and contribute significantly to decoding latency. To our knowledge, we are the first to study post-training quantization of recurrent states in GDN and KD

Read original source ↗ Open in workspace

recordType

paper

region

Global