SOURCE-LINKED INTELLIGENCE

Beyond Execution: Auditing Experimental Fidelity in LLM-Driven Scientific Research

arXiv · AI, language, vision and robotics · article · Aug 27, 2026 · UTC

LLM agents used for scientific experimentation must do more than generate executable code: they must implement the reference method faithfully, design experiments that test the paper's claims, and provide evidence supporting those claims. We show that agents often produce methodological hallucinations: silently reducing datasets or training budgets, replacing failed learning or generative components with lookup or oracle functions, or drawing conclusions from resource-limited settings where a method's claimed advantage disappears. To detect these failures, we introduce ABE-Ralph, a reference-a

Read original source ↗ Open in workspace

recordType

paper

region

Global