SOURCE-LINKED INTELLIGENCE

NumBench: Diagnosing Counting Failures in Text-to-Image Models

arXiv · AI, language, vision and robotics · article · Aug 28, 2026 · UTC

Text-to-image (T2I) models often generate the wrong number of objects, yet existing benchmarks are too small or weakly controlled to explain why. We introduce \textbf{NumBench}, a benchmark of 640{,}000 prompts spanning 1{,}600 categories and counts from 1 to 100. Its factorial design varies object composition, spatial guidance, and appearance conditions while balancing counts and category exposure. We also develop a process model in which requested instances compete for a finite set of resolvable image regions. The model predicts a near-quadratic collision deficit at low occupancy and shows h

Read original source ↗ Open in workspace

recordType

paper

region

Global