Energy‑Based Transformers Scale Faster and Think Deeper Than Standard Models
Energy-Based Transformers (EBTs) replace next‑token prediction with a learned verification function, letting models refine outputs via gradient‑based energy minimization. The paper shows EBTs scale up to 35% faster than standard Transformers and improve out‑of‑distribution performance by up to 29%, heralding a new, modality‑agnostic “System 2” thinking paradigm.