SOURCE-LINKED INTELLIGENCE

One Form to Transfer Them All: Pretraining Multilingual Language Models Beyond Native Orthography

arXiv · AI, language, vision and robotics · article · Aug 26, 2026 · UTC

Multilingual language models transfer knowledge across languages through shared subword vocabulary, a mechanism that breaks down when related languages use different writing systems. Prior work addresses this via script equalization (romanization or IPA transcription), but direct comparisons are rare; the focus has been on encoder-only models, with most work adapting existing pretrained models. We systematically compare different input representations in autoregressive multilingual pretraining, comparing orthographic text, IPA, and romanization in a controlled setup across three scales (467M,

Read original source ↗ Open in workspace

recordType

paper

region

Global