SOURCE-LINKED INTELLIGENCE

Predicting Program Exit Code with LLMs and Programming Language Semantics

arXiv · AI, language, vision and robotics · article · Sep 1, 2026 · UTC

Large language models (LLMs) have shown proficiency in various software engineering tasks, such as code generation and translation. However, a key limitation in their performance may be their (lack of) understanding of programming-language semantics. Even when explicit semantics are given, it remains unclear whether LLMs apply those rules or lean on priors learned during pre-training instead. We study if LLMs lean on priors or given semantics with a novel task--Program Executability Prediction (PrEx)--that asks models to predict whether a program is semantically valid or invalid (and, if inval

Read original source ↗ Open in workspace

recordType

paper

region

Global