SOURCE-LINKED INTELLIGENCE

KITE: KV-Invariant Transformer Expansion for Efficient Agentic LLM Scaling

arXiv · AI, language, vision and robotics · article · Sep 23, 2026 · UTC

Scaling a language model is not only a question of final quality: the architectural choice determines how much computation is spent during training, prompt processing, and autoregressive decoding to achieve certain model quality. An ideal model architecture should lower all above computation costs to facilitate scaling to a larger model, while ensure the larger model indeed outperforms smaller baselines. We introduce KV-Invariant Transformer Expansion (KITE), a scaling paradigm that achieves this goal. It trains the model from a smaller size to a larger size (i.e., saving training costs via up

Read original source ↗ Open in workspace

recordType

paper

region

Global