SOURCE-LINKED INTELLIGENCE

SharedSAE: One Feature Dictionary Across Language Models

arXiv · AI, language, vision and robotics · article · Sep 3, 2026 · UTC

Sparse autoencoders (SAEs) are widely used to interpret language model activations, but SAE training and latent labelling are typically repeated for every model. Here, we show that a single shared SAE can replace a collection of dedicated per-model SAEs. Our method, SharedSAE, combines a shared dictionary with model-specific encoder-decoder pairs. Unlike the closest prior method, which discards activation magnitudes and requires all models at inference, SharedSAE instead normalizes only selection scores, preserving magnitudes, and uses model dropout for single-model inference. We train SharedS

Read original source ↗ Open in workspace

recordType

paper

region

Global