SOURCE-LINKED INTELLIGENCE

Transformer Heads Looking for Order

arXiv · AI, language, vision and robotics · article · Sep 22, 2026 · UTC

In this note, we show that the problem of checking, whether a sequence of bits is ordered, is not doable by 1-head 1-layer transformers but is doable by a 2-head 1-layer transformer. Unlike similar previous results, our results assume the model where transformers have an output MLP.

Read original source ↗ Open in workspace

recordType

paper

region

Global