J-Lens loses to the simpler Logit Lens on GPT-2
A replication on GPT-2 small and medium found the J-Lens failed its core test: it beat the classic Logit Lens at 0 of 11 layers on small and just 1 of 23 on medium. The accompanying code and analysis reproduce the medium-model result, then identify a likely culprit: Jacobian transport amplifies attention-sink and document-register structure instead of token content. That is a useful warning for interpretability work: a more elaborate readout is not automatically a better one.