One SAE Dictionary Makes Model Interpretability Portable
Interpretability usually starts over for every language model: new SAE, new latent labels, new alignment work. SharedSAE trains one dictionary across four unrelated 1B-scale models, preserving 96.6% of dedicated SAEs’ explained variance while producing 1.8× stronger cross-model latent correlations. Freeze the dictionary, then adapt it to new models without rebuilding the vocabulary of features.