Energy‑Based Transformers Scale Faster and Enable System 2 Thinking
Energy‑Based Transformers (EBTs) replace direct prediction with a verification function that scores input–output pairs, letting models infer answers via gradient‑descent energy minimization. Across text and vision, EBTs train 35% faster than Transformer++ and improve inference by up to 29% on language tasks, while beating diffusion transformers on image denoising, especially on out‑of‑distribution data.