SOURCE-LINKED INTELLIGENCE

Apples to Apples? Towards Comparable Crosslingual Language Model Evaluation

arXiv · AI, language, vision and robotics · article · Aug 25, 2026 · UTC

Crosslingual evaluation of language models that enables fair comparisons remains a fundamental challenge in multilingual NLP. Existing studies adopt a variety of downstream tasks and intrinsic metrics with different theoretical justifications, yet there has been little empirical investigation into whether these approaches yield meaningful crosslingual conclusions. We systematically examine crosslingual evaluation approaches using controlled monolingual language models trained on parallel data with varying tokenizer vocabulary sizes and model sizes, and further validate our findings on multilin

Read original source ↗ Open in workspace

recordType

paper

region

Global