SOURCE-LINKED INTELLIGENCE

Continual Learning Mechanisms Compose for Long-Horizon Memorization

arXiv · AI, language, vision and robotics · article · Sep 7, 2026 · UTC

Language models may need to internalize information that arrives over time and retain it through many subsequent updates. To study this challenge, we introduce long-horizon memorization, a setting in which a model learns 100 query-answer tasks through continual supervised fine-tuning without retaining earlier training examples or receiving task identifiers at inference. Sequential updates cause catastrophic forgetting, and no single continual learning mechanism we evaluate maintains strong retention at this horizon. We hypothesize that mechanisms addressing complementary sources of forgetting

Read original source ↗ Open in workspace

recordType

paper

region

Global