SOURCE-LINKED INTELLIGENCE
The COUNT: Counting N-grams Unlocks Scaling on Host Memory and Training-Free Adaptation
The scaling relationship of compute and performance for language models (LMs) has been extensively studied; more recently, models are being designed to scale memory use, as well. In particular, adding embeddings for n-grams scales memory separately from compute use, because each pass only processes individual embedding rows, which are much smaller than the storage used for the whole embedding table. However, existing methods train embeddings from scratch, which needs expensive GPU memory for fast reading and writing. Meanwhile, the ordinary memory of the host system is cheap and underutilized
Read original source ↗ Open in workspace
- recordType
- paper
- region
- Global
Evidence & attribution
- arXiv · AI, language, vision and robotics · 2026-10-14T16:38:19.000Z
First collected: 2026-10-15T15:42:31.647Z. This is not the publication date.