SOURCE-LINKED INTELLIGENCE

$λ$-Controlled GRPO: Turning Flow-Matching Ratio Instability into a Budgeted Resource

arXiv · AI, language, vision and robotics · article · Sep 18, 2026 · UTC

Reinforcement learning is increasingly used to align image generators with reward signals, and Flow-GRPO recently extended this paradigm to flow-matching models by treating the denoising sampler as a stochastic policy that can be optimized from reward feedback. Training in this setting is unstable in a way specific to multi-step denoising: the policy update changes systematically across denoising steps, with importance ratios drifting below one, becoming increasingly dispersed, clipping at different rates, and leaving fewer usable samples late in training. Prior work treats these effects as se

Read original source ↗ Open in workspace

recordType

paper

region

Global