Hidden Interaction Effects Break Activation Patching’s Causal Estimates
A fresh causal analysis shows activation patching’s natural indirect effect conflates true component influence with interaction effects from other parts of the network. Those hidden interactions can hide or exaggerate a component’s role, calling into question many prior mechanistic interpretability results and suggesting new diagnostic metrics.