SOURCE-LINKED INTELLIGENCE

Recall Is Not Protection: Evaluating Safety Monitors Against Model Compliance

arXiv · AI, language, vision and robotics · article · Sep 5, 2026 · UTC

Safety monitors screen prompts sent to deployed language models, flagging harmful requests so they are never answered. They are evaluated by recall against harmfulness labels, but a catch only prevents harm if the model would otherwise have complied. We measure the difference directly: we sample repeated responses from the target model, call a harmful prompt \emph{elicitable} if the model complies at least once, and report monitor recall separately on elicitable and non-elicitable prompts. Across six monitor configurations and three model families, spanning activation probes, fine-tuned text g

Read original source ↗ Open in workspace

recordType

paper

region

Global