SOURCE-LINKED INTELLIGENCE

Modular Norm RandOpt: Population-Efficient Ensembling through Architecture-Aware Perturbations

arXiv · AI, language, vision and robotics · article · Sep 22, 2026 · UTC

RandOpt samples weight-perturbed language models and ensembles top-ranked candidates through plurality voting, but its global perturbation scale ignores heterogeneous module geometry. We propose \mbox{\textbf{\emph{Modular Norm RandOpt}}}, an architecture-aware sampling method using module-wise natural norms and calibrated scales while preserving selection and voting. It outperforms RandOpt using $3\times$ fewer candidates on Countdown and at least $12\times$ fewer on GSM8K, with corresponding wall-clock savings. Evaluations across seven tasks and three Qwen scales ($0.5$B--$3$B) show higher m

Read original source ↗ Open in workspace

recordType

paper

region

Global