SOURCE-LINKED INTELLIGENCE

Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness

arXiv · AI, language, vision and robotics · article · Sep 3, 2026 · UTC

How do the methods used to train language models to refuse harmful requests shape how that refusal actually works inside the model? We compare three post-training methods - supervised fine-tuning, reasoning-augmented fine-tuning (training on reasoning chains that justify a safety decision), and preference optimization (ORPO) - across three architecturally distinct models (Llama-3.1-8B, Gemma-2-9B, Qwen3-8B). We find that training method, not just data, reshapes how refusal is computed internally: reasoning-augmented training consistently produces a distinct kind of refusal computation, visible

Read original source ↗ Open in workspace

recordType

paper

region

Global