Production agents can cut evaluation to 38.5% of a full run
Teams maintaining production LLM agents can evaluate them with 200 questions, 38.5% of a full benchmark, while keeping mean absolute error at 1.03 percentage points using multidimensional 2PL adaptive testing. In practice, the authors deployed simpler difficulty-stratified fixed subsets, which transferred without recalibration to five other agent families and stayed stable with calibration windows as short as one day.