SOURCE-LINKED INTELLIGENCE

JEV vs. LLMs as Rubric Judges: Cheaper, Faster, and Wrong in the Same Places

arXiv · AI, language, vision and robotics · article · Sep 24, 2026 · UTC

We ask whether Jev, a typed classifier that returns probabilities over permitted answers without generating text, can replace an LLM rubric judge. We compare it with three flash-tier LLM judges on nine panels drawn from seven benchmarks, giving every judge identical criterion texts. Jev's accuracy differs significantly from an LLM judge's in only 8 of 27 paired comparisons, ahead mostly on binary criteria and behind only on graded ones, and most of the other comparisons are inconclusive. Summed over the nine panels, the LLM judges, called once per criterion, cost 29 to 325 times as much as Jev

Read original source ↗ Open in workspace

recordType

paper

region

Global