SOURCE-LINKED INTELLIGENCE

Improving Language Identification for Code-Switched Utterances with Integer Linear Programming

arXiv · AI, language, vision and robotics · article · Sep 4, 2026 · UTC

Automatic identification of code-switched (CS) utterances remains a challenge for language identification (LID) systems, causing such texts to be underrepresented in the training data of Large Language Models. In this paper, we revisit MaskLID, a state-of-the art approach for CS identification, which requires no training and detects arbitrary language combinations. We make three main contributions: (a) we reveal, and address, a major issue of MaskLID: its overreliance on word-level language association scores; (b) we reformulate the underlying optimization algorithm as an Integer Linear Progra

Read original source ↗ Open in workspace

recordType

paper

region

Global