SOURCE-LINKED INTELLIGENCE

Some Tokens Behave like Magnets: Revealing Linguistic Organization in the Layers of Language Models

arXiv · AI, language, vision and robotics · article · Sep 4, 2026 · UTC

We identify a special group of token vectors inside large language models (LLMs), which we term magnetic vectors, that organize the surrounding tokens by either attracting or repelling them. Particularly, tokens pointing the same way as an attracting magnet are elongated; tokens pointing the same way as a repelling magnet are compressed. Just as physical magnets pull or push away the iron filings around them, these vectors organize their surroundings through two opposing polarities. Moreover, we identify a statistically significant pattern in linguistic category where function words consistent

Read original source ↗ Open in workspace

recordType

paper

region

Global