NextLat gives transformers compact belief states and faster decoding
Microsoft Research introduces Next-Latent Prediction (NextLat), an auxiliary loss that trains transformers to forecast their own next hidden state. This forces the model to build compact belief-state representations, yielding better generalization, higher downstream accuracy, and up to 3.3x faster decoding on language tasks.