Looping an already trained model pays off only for the 0.8B size tested
Looping re-runs a block of layers so each token passes through it more than once, keeping parameter count fixed while every token pays the extra compute. Across Qwen3 and Qwen3.5 models from 0.8B to 9B, only the looped 0.8B model beat both the same model trained without the loop and the best unlooped use of the same compute. The benchmarks had room to improve: the unlooped 9B model scores up to 13 points higher than the 4B.