SOURCE-LINKED INTELLIGENCE

PatchBench: Evaluating AI Agents for Vulnerability Patching

arXiv · AI, language, vision and robotics · article · Sep 3, 2026 · UTC

AI agents have recently demonstrated strong performance in automated vulnerability patching. However, existing evaluations often validate a patch only by testing whether the provided Proof-of-Concept (PoC) input still triggers a crash. This leaves two key threats to validity: agents may reproduce memorized historical developer patches, or they may generate surface-level fixes that only suppress the reported crash. We study these concerns for C/C++ vulnerability patching. We introduce a patch similarity metric to detect memorized patches. On average, 25% of the agent patches exhibit substantial

Read original source ↗ Open in workspace

recordType

paper

region

Global