SOURCE-LINKED INTELLIGENCE

BenchBench-Protocol: Evaluating Real-World Wet-Lab Protocol Reasoning and Modification

arXiv · AI, language, vision and robotics · article · Aug 24, 2026 · UTC

We introduce BenchBench-Protocol, a benchmark for large language models of 149 protocol-modification tasks recovered from modifications that scientists made to published protocols during real experimental work. Adapting a published protocol to a new experiment is a routine task for a wet-lab scientist, and a correct modification requires accounting for prior choices and downstream steps. Recent life-science benchmarks have moved toward open-ended, rubric-graded tasks, but tasks are typically elicited from experts rather than reconstructed from real-world modifications. BenchBench-Protocol task

Read original source ↗ Open in workspace

recordType

paper

region

Global