SOURCE-LINKED INTELLIGENCE

Dissecting Hierarchical Reasoning Models: A Mechanistic Study

arXiv · AI, language, vision and robotics · article · Aug 30, 2026 · UTC

We study Hierarchical Reasoning Model (HRM), a representative hierarchical Transformer-based latent reasoning model with many variants, on Sudoku, Maze, and ARC-AGI-2. We mechanistically understand how HRM reasons and what information it encodes. Our analyses compare HRM against Transformer baselines with and without recurrent modules, apply causal interventions on recurrent states, and utilize linear probes against random-direction ablations, as well as sparse autoencoders with feature ablations. Our results reveal several key findings: recurrent models outperform one-pass baselines, while si

Read original source ↗ Open in workspace

recordType

paper

region

Global