SOURCE-LINKED INTELLIGENCE

World Modeling in Transformers

arXiv · AI, language, vision and robotics · article · Sep 18, 2026 · UTC

Behavioral failures can make a transformer appear to lack a world model even when it has learned faithful representations of its environment. We demonstrate this in TaxiGPT, a transformer trained on random walks through Manhattan whose failures have been interpreted as evidence of an incoherent internal map. Through mechanistic analysis and causal interventions, we show that the model represents intersections and streets, tracks its position, and uses a goal compass to navigate. We trace its failures to interference between superposed intersection features, which disrupts localization within t

Read original source ↗ Open in workspace

recordType

paper

region

Global