Transformers lock in answers at a single “Hard Decision Layer,” boosting accuracy
Researchers found that during multiple‑choice QA, transformer models abruptly stabilize answer rankings at a specific internal layer—the Hard Decision Layer. This layer consistently yields the highest accuracy boost (up to +0.61 on CommonsenseQA) and persists across models and fine‑tuning, offering a clear mechanistic window into when models commit to predictions.