SOURCE-LINKED INTELLIGENCE

Scaling Attention Head Analysis via Gradient-Based Attribution in Context-Aware Machine Translation

arXiv · AI, language, vision and robotics · article · Sep 23, 2026 · UTC

In this paper, we introduce a gradient-based head attribution strategy where the Token-level Max-Margin loss is backpropagated to the attention maps. This framework enables a large-scale causal analysis of attention heads, making it suitable for LLMs. We evaluate our method on the task of disambiguation in Context-aware Machine Translation, where we analyze 50 phenomena across 4 models and 4 language directions. We empirically show the alignment of our method with the effects of increasing the attention scores of token-to-token relations on three models and two language directions, ensuring th

Read original source ↗ Open in workspace

recordType

paper

region

Global