SOURCE-LINKED INTELLIGENCE

CraftBench-UE: Deterministic Evaluation for Coding Agents in Unreal Engine

arXiv · AI, language, vision and robotics · article · Sep 19, 2026 · UTC

Building gameplay features in a game engine requires more than code, as code that compiles and runs does not necessarily implement the requested gameplay. We introduce CraftBenchUE, an evaluation harness that runs agents in an isolated Unreal Engine environment, reconstructs their saved submissions in fresh projects, and applies deterministic build, asset, and runtime checks without an LLM judge. Based on the harness, we built a benchmark consisting of 70 tasks spanning C++ source, Blueprint assets, and editor scripting. We evaluate seven models under two editor-tool configurations, with a fil

Read original source ↗ Open in workspace

recordType

paper

region

Global