SOURCE-LINKED INTELLIGENCE

The Blindness of Document-Level Translation Evaluation

arXiv · AI, language, vision and robotics · article · Sep 5, 2026 · UTC

Document-level machine translation (MT) evaluation extends segment-level protocols by presenting full documents to annotators, on the assumption that such presentation elicits document-level judgments. We test this assumption with a counterfactual condition (MIX) in which each document combines segments drawn from different systems, preserving document-level presentation while breaking cross-segment consistency. Across 18,420 expert Englis-to-Korean annotations and 14 automatic metrics, scores, system rankings, and error annotations are statistically equivalent between coherent and incoherent

Read original source ↗ Open in workspace

recordType

paper

region

Global