SOURCE-LINKED INTELLIGENCE

Feature Superposition in Neural Networks: From Theory to Practice

arXiv · AI, language, vision and robotics · article · Sep 6, 2026 · UTC

Superposition refers to neural networks representing more features than they have dimensions. It offers a possible explanation for polysemantic neurons and motivates methods for recovering interpretable features from neural activations. Theoretical models typically start with a given set of input features and assumptions about how their values vary across inputs, then study how a network encodes those values in a lower-dimensional hidden representation. Empirical work, by contrast, seeks to identify the features encoded in trained networks and determine their role in computation. In this surve

Read original source ↗ Open in workspace

recordType

paper

region

Global