SOURCE-LINKED INTELLIGENCE

Reliability, validity, and diagnostic evidence for multi-model LLM short-answer scoring

arXiv · AI, language, vision and robotics · article · Sep 6, 2026 · UTC

Large language models (LLMs) are increasingly used or proposed for educational scoring, but single-model and single-run evaluations provide limited evidence for assessment use. Short-answer scoring requires evidence about reliability, validity, severity, diagnostic value, and failure cases. This study evaluated repeated multi-model OCG-PRES guided LLM scoring for short-answer assessment. The analysis used 996 SciEntsBank responses. GPT, DeepSeek, and Qianwen each scored every response across three independent runs using five OCG-PRES dimensions: concept coverage, relation accuracy, reasoning c

Read original source ↗ Open in workspace

recordType

paper

region

Global