SOURCE-LINKED INTELLIGENCE

Neural Spectral Capacity: Measuring and Designing Architectures from Network Specification Alone

arXiv · AI, language, vision and robotics · article · Sep 19, 2026 · UTC

Modern Transformer design and compression both reduce to allocating capacity under a budget. The standard scalars for these decisions, #Params and #FLOPs, capture size and compute but not architectural structure: two architectures with identical parameter budgets but different depth-width, head, or FFN allocations receive identical scores yet behave differently. We propose Neural Spectral Capacity (NSC), a closed-form scalar grounded in the singular-value spectrum of each weight matrix. Under standard random initialization, the Marchenko-Pastur law renders NSC computable from the architectural

Read original source ↗ Open in workspace

recordType

paper

region

Global