SOURCE-LINKED INTELLIGENCE

Patterning in Practice: Debiasing Reward Models with Susceptibilities

arXiv · AI, language, vision and robotics · article · Sep 1, 2026 · UTC

Reward models trained on human preferences are known to suffer from length, formatting, and other stylistic biases. In this paper we use patterning, which reweights each preference pair according to its measured effect on posterior expectation values of benchmark losses (its susceptibility), to debias a Gemma 2 9B Instruct reward model trained on Skywork-Reward-Preference v0.2. We obtain $+14.2 \pm 1.2$ pp on RM-Bench Hard, the split where style cues point against correctness (mean $\pm$ s.e.\ over 5 seeds), with overall RM-Bench accuracy preserved, comparable to the strongest Hard-split gain

Read original source ↗ Open in workspace

recordType

paper

region

Global