SOURCE-LINKED INTELLIGENCE

What Attention Recalls and Recurrence Controls in Hybrid Language Models

arXiv · AI, language, vision and robotics · article · Sep 3, 2026 · UTC

Hybrid language models combine attention with a fixed-size recurrent state, but the role of each channel remains unclear. We introduce two cache-level interventions. Split-prefill keeps only the KV cache or only the recurrent state from a prefilled context, then generates an answer. State-swap pairs the KV cache from one context with the recurrent state from another in a single forward pass. On Qwen3.5 and Falcon-H1, the two channels split sharply by function. Exact retrieval survives only through attention (64-98% of full accuracy) and collapses to zero through recurrence. Output language and

Read original source ↗ Open in workspace

recordType

paper

region

Global