SOURCE-LINKED INTELLIGENCE

From Spectra to Joint Schedules in LLM Pre-training: 3+3(+2) Scaling-Law Regimes

arXiv · AI, language, vision and robotics · article · Sep 30, 2026 · UTC

Power-law learning curves are often treated as fixed properties of a model and its data, although learning-rate and batch-size schedules can change the observed loss. We study this dependence in noisy online SGD with linear random features. Conditional on the representation, an exact Volterra equation separates two response components: a forcing term that propagates unresolved target error and a memory kernel that propagates stochastic-error injections. We prove that either component follows a power law if and only if its cumulative weighted spectral mass has the corresponding low-spectrum sca

Read original source ↗ Open in workspace

recordType

paper

region

Global