SOURCE-LINKED INTELLIGENCE

Tracking States or Tracking Cosets? An Algebraic Account of Learned State Tracking

arXiv · AI, language, vision and robotics · article · Sep 24, 2026 · UTC

State tracking requires composing a sequence of updates, but accuracy alone does not reveal what a model has learned. We study neural networks trained to predict the running product of group elements. We identify quotient solutions in Transformers, where models recover the quotient class while predicting nearly uniformly among its members. The reciprocal of class size predicts partial accuracy without a fitted parameter, extending parity-based accounts to non-parity quotients. Our baseline Transformers' predictions change little under prefix reordering beyond the exact-tracking frontier. We pr

Read original source ↗ Open in workspace

recordType

paper

region

Global