SOURCE-LINKED INTELLIGENCE

ToolGate: An Executable Acceptance Pipeline for Tool-Dependent Scientific Benchmark Construction

arXiv · AI, language, vision and robotics · article · Sep 2, 2026 · UTC

Scientific benchmarks are commonly built by domain experts who write tasks and cross-check one another's work, or who adapt existing material from textbooks, published papers, and online resources. These routes can produce strong evaluations, but they require substantial per-item labor. Language models can reduce this repeated work by proposing candidates quickly. The remaining problem is acceptance. We target scientific questions whose answers require computations with specialist software rather than unaided reasoning alone. A candidate is invalid if its script fails or returns a different an

Read original source ↗ Open in workspace

recordType

paper

region

Global