SOURCE-LINKED INTELLIGENCE

Load-Bearing Context: The Question Damage Score for Evaluating Context Reliance in Linguistic Reasoning

arXiv · AI, language, vision and robotics · article · Aug 27, 2026 · UTC

Determining whether large language models derive answers from context or prior knowledge remains a fundamental challenge. Self-contained linguistic olympiad puzzles provide a controlled setting where all answers derive solely from expert-designed context examples without external knowledge. Removing individual context examples can eliminate information needed for specific questions while leaving the rest of the puzzle unchanged. We leverage this to introduce a diagnostic framework for analyzing individual context examples. Using 53 UK Linguistics Olympiad puzzles, we generate two modified vari

Read original source ↗ Open in workspace

recordType

paper

region

Global