SOURCE-LINKED INTELLIGENCE

One Capability or Many? Testing the Economic Validity of Frontier AI Evaluation

arXiv · AI, language, vision and robotics · article · Aug 29, 2026 · UTC

Frontier-model leaderboards now rank systems based on economic benchmarks, tests of how well models carry out professional tasks from software engineering to banking workflows, and those rankings inform what organisations buy, what regulators scrutinise, and expectations of how work will change. Whether such benchmarks measure a capability distinct from general test-taking, or re-express the one axis along which every benchmark rises as models improve, is a question of construct validity that has not yet been studied. We test it on a hash-pinned leaderboard snapshot of 421 model configurations

Read original source ↗ Open in workspace

recordType

paper

region

Global