OmniLens cuts LLM interpretability cost, scales hidden-state analysis to 70B models
The new OmniLens framework replaces quadratic‑growth affine translators with low‑rank versions and a Subset‑KL training trick, slashing parameters by up to 98% and memory by 70%. This lets researchers run 482 lenses on LLaMA‑3.3‑70B, exposing effective interventions beyond attention heads while reproducing prior findings at far lower cost.