SOURCE-LINKED INTELLIGENCE

The Sequential Price of Continual Learning

arXiv · AI, language, vision and robotics · article · Aug 30, 2026 · UTC

Sequential task updates are fundamental to continual learning, but their recency bias can impose a lasting performance cost. We study this cost in an overparameterized linear-regression model with i.i.d. task sampling. We prove that distribution-level forgetting and population loss converge to the same stationary limit. This common limit separates exactly into the intrinsic loss asymptotically attained by joint training and an additional sequential price, and in more homogeneous task geometries the two terms coincide, making the total loss twice that of joint training. We further analyze fixed

Read original source ↗ Open in workspace

recordType

paper

region

Global