SOURCE-LINKED INTELLIGENCE

Mahalanobis-Based Multi-Head Attention for Complex State Propagation

arXiv · AI, language, vision and robotics · article · Aug 25, 2026 · UTC

In this paper, we propose \textbf{Mahalanobis-Based Multi-Head Attention} (MHA-CSP), a novel attention mechanism that replaces the standard dot-product with a \textbf{Mahalanobis distance-based RBF kernel}, which effectively computes attention in an infinite-dimensional feature space without increasing the parameter count. Crucially, the positive definiteness of the Mahalanobis distance enables a \textbf{direct construction of Tree Attention}: attention scores are built directly from accumulated distances, with a LogSumExp correction that rectifies the raw distance by subtracting the log-sum o

Read original source ↗ Open in workspace

recordType

paper

region

Global