SOURCE-LINKED INTELLIGENCE

Evolutionary Soups: Evolving Mixture-of-Experts for Multi-Objective LLM Alignment

arXiv · AI, language, vision and robotics · article · Aug 30, 2026 · UTC

Large language models are increasingly required to generate responses that satisfy multiple competing objectives. Since optimal trade-offs depend on both user preferences and input prompts, controllable multi-objective generation must dynamically adapt models at inference time without retraining. To address this, we propose Evolutionary Soups, a mixture-of-experts framework for fine-grained generation control, with gating networks trained via an evolutionary algorithm. The per-layer gating networks dynamically produce expert-merging coefficients from hidden-state representations, while the evo

Read original source ↗ Open in workspace

recordType

paper

region

Global