SOURCE-LINKED INTELLIGENCE

The Changing Geometry of Grammar: Dimensionality and Neighborhood Reorganization across Transformer Layers

arXiv · AI, language, vision and robotics · article · Aug 25, 2026 · UTC

Transformer representations describe trajectories through high-dimensional vector spaces, which are shaped dynamically as tokens incorporate relational context across layers. Such data tend to concentrate on lower-dimensional sub-manifolds, a form of compression quantified by the Intrinsic Dimensionality (ID), the minimum number of independent variables needed to represent them without significant information loss. In this work, we ask whether the grammatical role of tokens, as marked by their part-of-speech (PoS) tag, shapes the local geometry of this manifold. To this end: (1) We investigate

Read original source ↗ Open in workspace

recordType

paper

region

Global