A Transformer That Turns Every Token Into More Depth
Most models spend a fixed amount of computation on each token. Recurrent Looped Transformer instead carries decoder state and layerwise attention caches forward, so a 48-layer decoder traverses 48 blocks per token and builds a temporal path of 48t blocks across a sequence. The design is unusually ambitious, but the page is explicit: reasoning gains, hardware efficiency, and RL scaling are still unproven.