SOURCE-LINKED INTELLIGENCE

Layerwise Decoupling for Stable Structured Sparsification of Fully Connected Layers

arXiv · AI, language, vision and robotics · article · Sep 17, 2026 · UTC

We propose a decoupled, layerwise method for structurally sparsifying the fully connected layers of pretrained neural networks. Rather than penalizing all layers jointly, our approach extracts shallow two-layer subnetworks, normalizes the inner weights, and applies a structured group penalty to the outer weight matrix of each block, processing layers sequentially to prune neurons and reduce the width of each layer. We prove that the constrained decoupled objective is equivalent at optimality to a specific joint penalty on the inner and outer weights, for any positively homogeneous activation,

Read original source ↗ Open in workspace

recordType

paper

region

Global