AIIC AI Intelligence Centre

SOURCE-LINKED INTELLIGENCE

Same Simulation, Different Evidence: Where Retrieved Tool Output Costs LLM Agents Accuracy

arXiv · AI, language, vision and robotics · article · Oct 19, 2026 · UTC

LLM agents that compute the right answer can still answer wrongly. Agent harnesses increasingly keep long tool outputs out of the context window, saving them to files that the agent searches or reads on demand. We ask where this costs accuracy with a real scientific tool: PHREEQC, a widely used geochemistry simulator from the USGS, whose reports reach 846.8k characters and whose determinism lets every printed value be traced. Agents use it to answer 158 execution-verified multiple-choice items. When agents read the simulator's report through a section index rather than in context, GPT-5.1 and

Read original source ↗ Open in workspace

recordType
paper
region
Global

Evidence & attribution

First collected: 2026-10-20T12:22:22.327Z. This is not the publication date.