SOURCE-LINKED INTELLIGENCE

Conformity Breaks Conformal Prediction

arXiv · AI, language, vision and robotics · article · Sep 3, 2026 · UTC

A conformal certificate can be valid when an LLM answers alone and invalid when the same LLM sees peers that unanimously assert a wrong answer. The question is unchanged; the model's score for the correct answer changes. We call this a score-mechanism shift: clean calibration certifies how the model scores answers alone, but not how it scores them under peer pressure. We show that this shift silently breaks conformal prediction in multi-agent LLM systems. Across open-weight models and multiple-choice QA tasks, coverage falls from a calibrated 90% to 74% under unanimous-wrong peers at the stand

Read original source ↗ Open in workspace

recordType

paper

region

Global