SOURCE-LINKED INTELLIGENCE

Rethinking Vision Architectures with Gated Linear Attention and KAN

arXiv · AI, language, vision and robotics · article · Sep 18, 2026 · UTC

Vision Transformers allocate most parameters to multi-layer perceptrons (MLPs) for channel mixing, while token interactions usually rely on quadratic multi-head self-attention (MHSA). Linear attention reduces sequence complexity to O(N), but remains coupled with the same fixed-activation MLP as softmax Transformers. Kolmogorov-Arnold Networks (KANs) instead place learnable univariate maps on edges, yet prior vision KANs keep MHSA or omit attention entirely. We introduce LKAT (Linear Kolmogorov-Arnold Transformer), an isotropic ViT encoder that couples chunkwise Gated Linear Attention (GLA) wit

Read original source ↗ Open in workspace

recordType

paper

region

Global