SOURCE-LINKED INTELLIGENCE

Matrix AdaGrad: Row-wise and Column-wise Adaptive Subgradient Methods

arXiv · AI, language, vision and robotics · article · Sep 18, 2026 · UTC

Adaptive optimization methods such as AdaGrad and Adam are widely used in modern neural-network training, but their adaptive scaling is primarily designed for vector-valued parameters and does not explicitly exploit matrix structure. Recent matrix-aware optimizers demonstrate the benefits of structured optimization, yet a general theoretical framework for deriving matrix-aware adaptivity comparable to that of AdaGrad remains lacking. In this work, we develop a general Online Mirror Descent framework with adaptive proximal functions for matrix-valued parameters, providing a principled approach

Read original source ↗ Open in workspace

recordType

paper

region

Global