SOURCE-LINKED INTELLIGENCE

Guardrail-Agnostic Societal Bias Evaluation in Large Vision-Language Models

arXiv · AI, language, vision and robotics · article · Aug 30, 2026 · UTC

We propose a societal bias evaluation method for large vision-language models (LVLMs) in the era of strong safety guardrails. Existing benchmarks rely on prompts that ask models to infer attributes of people in images (e.g., "Is this person a CEO or a secretary?"). However, we find that LVLMs with strong guardrails, such as GPT and Claude, often refuse these prompts, making evaluations unreliable. To address this, we change the prior evaluation paradigm by decoupling the task from the depicted person: instead of inferring person's attributes, we use prompts that do not ask about the person (e.

Read original source ↗ Open in workspace

recordType

paper

region

Global