SOURCE-LINKED INTELLIGENCE

Complementary Roles of Activation and Parametric Memory in Few-Shot Learning

arXiv · AI, language, vision and robotics · article · Sep 23, 2026 · UTC

At test time, large language models (LLMs) can encode historical information in activation memory (i.e., KV caches) and parametric memory (i.e., updated parameters). While activation memory is generally considered effective for factual recall and parametric memory for learning new tasks, their interplay remains unclear. In this work, we systematically investigate the role of memory in few-shot learning through controlled experiments. We find that activation memory is superior for recalling facts, whereas parametric memory does not consistently outperform activation memory in task learning. Mor

Read original source ↗ Open in workspace

recordType

paper

region

Global