SOURCE-LINKED INTELLIGENCE

Lngram v2: Latent N-Gram Memory with Interpretable Discrete Representations

arXiv · AI, language, vision and robotics · article · Sep 3, 2026 · UTC

Transformers lack a native lookup mechanism, requiring repeated dense computation to recognize and reuse local static patterns. Lngram v1 introduces tokenizer-independent conditional memory through discrete latent n-gram addressing, but its memory capacity is coupled with the backbone width, limiting scalability due to high parameter and activation costs. We propose Lngram v2, which decouples the number of routes, memory dimension, and backbone width, and introduces a context-aware grouped-query attention readout to scale memory capacity independently. A zero-value Sink and counterfactual surr

Read original source ↗ Open in workspace

recordType

paper

region

Global