AIIC AI Intelligence Centre

SOURCE-LINKED INTELLIGENCE

Deep Latents, Long Memory: Latent persistence explains temporal correlations in transformer activations

arXiv · AI, language, vision and robotics · article · Nov 13, 2026 · UTC

The hidden layers of Large Language Models (LLMs) are believed to encode representations of the data whose abstraction increases with depth. However, in the absence of an operational notion of abstraction, this statement is hard to test quantitatively. Here, we propose an alternative view, building on the principle that the correlations of a sequence reflect the structure of the process that generated it. We define the autocorrelation function of a layer as the average dot product between its activations at two context positions separated by a lag. Latent variables encoded persistently along t

Read original source ↗ Open in workspace

recordType
paper
region
Global

Evidence & attribution

First collected: 2026-11-16T10:41:26.071Z. This is not the publication date.