A Hidden Copying Head Exposes Limits of Grammar Interpretations in a Character Transformer
A character‑level transformer trained on Nietzsche reveals a single attention head that behaves like a copying mechanism, identified via eigenvalue analysis of its OV circuit. The author’s initial guesses about the head handling quotation or parenthesis grammar were disproved, highlighting how mechanistic interpretability can overturn its own hypotheses.