SOURCE-LINKED INTELLIGENCE

Can LLMs Catch a Rigged Backtest? A Clean-Control Calibration Benchmark

arXiv · AI, language, vision and robotics · article · Sep 23, 2026 · UTC

Backtest auditing is a calibration problem: high flaw recall is not useful when the model falsely flags matched clean strategies. We build a 96-item paired benchmark in which every flawed backtest has a clean control that holds strategy, dates, code style, labels, and reporting scaffold fixed while changing one methodology detail. A deterministic scorer separates flaw recall, clean-control false positives, evidence localization, and fix relevance. Over 1440 cached audits from four text endpoints, the primary DeepSeek auditor reaches 100.0\% closed and clean-aware code recall, but open prompts

Read original source ↗ Open in workspace

recordType

paper

region

Global