SOURCE-LINKED INTELLIGENCE

LLM-as-a-Demographic: Whom Sociodemographic Prompting Helps, and Whom It Hurts

arXiv · AI, language, vision and robotics · article · Aug 31, 2026 · UTC

Large language models (LLMs) are increasingly used as judges for subjective tasks, where annotators disagree and the relevant question is not only how accurate a judge is, but whose judgments it reproduces. Sociodemographic prompting conditions the judge on an annotator's demographic profile to align its judgments with the corresponding group's. We test whether this alignment emerges distributionally, comparing the predicted label distributions of 23 open-weight LLMs on three subjective tasks against those of real annotator groups, under three conditions: no demographic information, single-att

Read original source ↗ Open in workspace

recordType

paper

region

Global