Neural networks may be more interpretable than superposition suggested
Mechanistic interpretability may not need to untangle neural networks first: the apparent concept-mixing that made them look arbitrary was mostly recursive generalization, according to this analysis. It argues that superposition was the wrong explanation for InceptionV1’s confusing late layers, where architectural problems and weak tools made the network seem less coherent than it was. That leaves researchers able to study neurons and circuits directly, rather than relying first on sparse autoencoders to reconstruct hidden features.