Evaluation Pipelines Skew Sparse Autoencoder Interpretability Scores
Systematic tests show that autointerpretability scores for sparse autoencoders vary more with pipeline choices—prompt templates and scoring language models—than with the underlying model architecture. This undermines cross‑paper comparisons and calls for standardized evaluation protocols, including a variance decomposition tool and a reporting checklist.