SOURCE-LINKED INTELLIGENCE

Attention as a Routing Graph: Live Circuit Extraction from a Single Forward Pass

arXiv · AI, language, vision and robotics · article · Sep 21, 2026 · UTC

Finding circuits in language models usually means running many careful interventions. We try something simpler: treat attention as a routing map from one forward pass, keep a small set of routes that point toward the answer, and ask whether those routes actually matter. They often do. On induction and IOI (tasks where the "right" circuit is already known), ablating our extracted edges hurts the model much more than ablating a random set of the same size. We evaluate n=100 prompts per cell on GPT-2 Small, GPT-2 Medium, and Pythia-410M, with paired gap tests and bootstrap confidence intervals. T

Read original source ↗ Open in workspace

recordType

paper

region

Global