SOURCE-LINKED INTELLIGENCE

OmniJudge or OmniBias? Diagnosing Multimodal Judges through Balanced, Decoupled Lenses

arXiv · AI, language, vision and robotics · article · Aug 25, 2026 · UTC

Multimodal understanding models that can jointly judge text-to-image (T2I), text-to-video (T2V) and text-to-speech (TTS) generation are increasingly used as "OmniJudges" for evaluation and automatic annotation. How reliably they understand what they score remains unclear, since existing benchmarks and training data tend to overemphasize positive examples and to conflate distinct failure modes, so a judge may score well without recognizing failures while its capability gaps stay hidden. Motivated by this, we introduce D3-Omni, a balanced and decoupled benchmark for diagnosing fine-grained multi

Read original source ↗ Open in workspace

recordType

paper

region

Global