SOURCE-LINKED INTELLIGENCE

How Temporal Correlations Shape Memory in Linear Recurrent Neural Networks

arXiv · AI, language, vision and robotics · article · Aug 31, 2026 · UTC

The linear recurrent neural network (LRNN) is a simple model for studying how much memory a network builds up as it trains. For uncorrelated inputs, earlier work found that training itself settles the network between keeping the past and reacting only to the present. Real sequences are correlated, and we solve the learning dynamics exactly for correlated inputs. In the solution, keeping the past carries a cost. The whole effect of correlation lands on that cost. This cost reduces to the earlier one when inputs are uncorrelated and grows once they are positively correlated. Three findings follo

Read original source ↗ Open in workspace

recordType

paper

region

Global