Physics-inspired pruning makes 50% Llama cuts score far better
Multiverse Computing and CAI’s method improves 50%-compressed Llama-3.3-70B-Instruct by almost 23 MMLU points over the best competing block-removal method. It gives each transformer block a keep-or-remove variable, then models pairwise interactions between those choices as an energy-minimization problem instead of ranking blocks independently. That energy acts as a cheap proxy for benchmark quality, so many candidate pruning layouts can be ranked without benchmarking each one.