SOURCE-LINKED INTELLIGENCE

Your Transformer Can Hold Two Thoughts at Once: Evidence of Linear Superposition in LLMs

arXiv · AI, language, vision and robotics · article · Sep 24, 2026 · UTC

While Large Language Models (LLMs) rely on highly non-linear components, in this work we demonstrate that they exhibit fundamental linearity: when inputs from distinct text streams are linearly combined, the model outputs a superposition of the individual next-token distributions. We term this the \textit{Superposition Linearity Hypothesis}. We provide evidence that superposition is an intrinsic property of the Transformer architecture rather than an emergent consequence of training; in fact, we observe that it tends to diminish as pretraining progresses. However, we demonstrate that linearity

Read original source ↗ Open in workspace

recordType

paper

region

Global