NLA Thought Anchors Explain Why Autoencoders Favor Correct Predictions
The post examines Natural Language Autoencoders (NLAs) and shows they more often reproduce a model’s correct final answer when the originating activations produced a correct output. It links answer position, reconstruction loss, and degenerate NLA outputs to model correctness, offering new mechanistic insight into LLM predictions.