SOURCE-LINKED INTELLIGENCE

MineCEraft: Evaluating Language Models as Construction Engineers in the World of Minecraft

arXiv · AI, language, vision and robotics · article · Aug 28, 2026 · UTC

We introduce MineCEraft (Minecraft Construction Engineering Benchmark, pronounced mine-see-ee-raft), an easy-to-use, open-source benchmark designed to systematically evaluate the reliability and limitations of LLMs for construction tasks in Minecraft. The MineCEraft benchmark comprises 723 domain-expert hand-crafted natural-language instructions with programmatically verifiable evaluation, spanning 17 distinct task categories, providing a safe and controllable experimental environment for assessing LLMs' ability to perform realistic construction engineering tasks. With this benchmark, we condu

Read original source ↗ Open in workspace

recordType

paper

region

Global