SOURCE-LINKED INTELLIGENCE

COREM: Cosine-Relation Momentum Reshaping with Stateful Writeback

arXiv · AI, language, vision and robotics · article · Sep 18, 2026 · UTC

Matrix-valued optimizer states may contain relational structure that is not captured by treating their entries independently. We study whether relations within matrix-valued optimizer states can be exploited to improve optimization. To this end, we introduce a unit-relation-transform abstraction and instantiate it as COREM, a Cosine-Relation Momentum Reshaping method with stateful writeback. COREM partitions the momentum state into update units, computes cosine relations among them, and uses these relations to reshape the momentum before writing the transformed state back to the optimizer. Thi

Read original source ↗ Open in workspace

recordType

paper

region

Global