SOURCE-LINKED INTELLIGENCE

Speed Limit for Information Acquisition in Stochastic Learning Dynamics

arXiv · AI, language, vision and robotics · article · Sep 8, 2026 · UTC

Neural networks acquire internal representations through learning. In this work, we formulate stochastic gradient descent (SGD) as a Markovian stochastic process and derive a Fisher-information flow speed limit that bounds the rate at which trainable parameters can acquire information about latent variables in the data-generating process. The resulting inequality decomposes the information flow into drift and noise contributions, thereby quantifying the roles of deterministic learning forces and SGD-induced fluctuations from an information-theoretic perspective. We verify the bound in analytic

Read original source ↗ Open in workspace

recordType

paper

region

Global