AQuA: fixing the evaluator doesn't stop agent loops from gaming it
Freezing the judge is the standard fix for eval gaming. AQuA, a new preprint, tests it with sealed-sandbox LLM research agents for quant trading: data splits, labels, and evaluator stay locked, and agents act only through constrained factor expressions or config diffs. The defense mostly holds, with a held-out Sharpe up to 2.50 and positive returns every year from 2021 to 2025. But feedback from that fixed evaluator still steers the loop, so the judge remains a target of adaptation.