SOURCE-LINKED INTELLIGENCE

Landau theory of quenched criticality in linear in-context learning

arXiv · AI, language, vision and robotics · article · Aug 28, 2026 · UTC

In-context learning (ICL) allows a pretrained model to infer a new task from examples supplied in its prompt without updating its parameters. In linear models of ICL, the prediction error develops a double-descent singularity when the number of pretraining samples becomes comparable to the number of learnable parameters. We formulate this interpolation singularity as a critical phenomenon of a quenched disordered system. By comparing annealed and quenched descriptions of the same linear ICL model, we identify the connected sample-to-sample fluctuations of the learned parameters as the microsco

Read original source ↗ Open in workspace

recordType

paper

region

Global