SOURCE-LINKED INTELLIGENCE

Measuring the Assistant's Harmlessness Preferences on the User Turn

arXiv · AI, language, vision and robotics · article · Sep 20, 2026 · UTC

Post-training turns a general next-token predictor into a chat model with a persistent assistant persona. If that persona is a character the model plays only on its own turns, its preferences should govern what the assistant says, not what the model predicts other speakers will say. We test this boundary and find that it does not hold: a safety-relevant preference of the assistant---for harmless over harmful tasks---shapes the model's predictions even on the user's turn, where the assistant is not the one speaking. We find that this preference is small or near-zero in pretrained base models, t

Read original source ↗ Open in workspace

recordType

paper

region

Global