SOURCE-LINKED INTELLIGENCE

Training Alignment Auditors via Reinforcement Learning

arXiv · AI, language, vision and robotics · article · Aug 26, 2026 · UTC

Alignment auditing of frontier models increasingly relies on LLM auditors to surface undesirable behaviors at scale, but current automated auditors can struggle with coherent investigation and audit realism. In this work, we improve LLM auditors with reinforcement learning. In our best training environment, the policy investigates target models that potentially possess hidden behaviors planted via their system prompt. An LLM judge, which knows whether the target has a hidden behavior, holistically compares the policy's investigation to a reference investigation to determine the reward. With sy

Read original source ↗ Open in workspace

recordType

paper

region

Global