SOURCE-LINKED INTELLIGENCE

Complex-valued Phase-Coherent Transformers

arXiv · AI, language, vision and robotics · article · Sep 18, 2026 · UTC

Complex-valued Transformers have inherited softmax attention over the raw complex inner product. Outside natively complex domains this standard form stays near chance, and no complex attention had been shown to correct it. We show that the match must be a scaled cosine score: L2-normalise queries and keys, so the score reads their cosine similarity and ignores their magnitudes, and hold that score at order-one scale. With this the same models train on four diagnostic tasks under two different gates; without the normalisation they stay at chance on ListOps and Needle under both gates and fall f

Read original source ↗ Open in workspace

recordType

paper

region

Global