SOURCE-LINKED INTELLIGENCE

LLJ Cards: Best practices for the Use of LLMs as Judges

arXiv · AI, language, vision and robotics · article · Sep 21, 2026 · UTC

In recent years, large language models (LLMs) have emerged as a popular alternative for evaluation. Often referred to as LLMs as judges (LLJs), these systems have been widely adopted by researchers and practitioners across a broad range of measurement tasks, driven by their strong performance, scalability, and cost-effectiveness relative to human judgment. However, a growing body of work has shown that the use of LLJs raise concerns about their validity and reliability as evaluators. Existing efforts to address these challenges have largely focused on developing bias-mitigation techniques and

Read original source ↗ Open in workspace

recordType

paper

region

Global