SOURCE-LINKED INTELLIGENCE

KV-COBRA: KV Cache Compression via Co-Optimized Bit-Rank Allocation

arXiv · AI, language, vision and robotics · article · Sep 21, 2026 · UTC

What limits KV-cache compression at extreme bit-rates? We argue that it is not the choice of compression scheme, but how its budget is allocated across attention heads. Existing methods apply rank and bit-width uniformly, ignoring that each head has a different optimal mix of rank truncation and quantization. We show that co-optimizing rank and bit-width per head, using only standard low-rank projection and scalar quantization, dominates uniform allocation, with the largest gains at low bit-rates. Our method, KV-COBRA (Co-Optimized Bit-Rank Allocation), formalizes this as a resource-allocation

Read original source ↗ Open in workspace

recordType

paper

region

Global