LIFT lets language models carry hidden state between generation steps
LIFT gives language models a second route between generation steps, so they can carry forward internal information instead of passing everything through the decoded token. During training, each token is paired with a state from a pretrained model, and the new model learns to predict both the next token and next state in parallel. Across 135M to 1B parameters, the paper reports better language modeling, reasoning, and procedural-task results than standard Transformers under the same token budget.