In 31.1% of agent runs, tasks succeed despite a failed harness check
TraceTrial-Bench grades agent harnesses, the software around a model that handles context, tools and checks, separately from the model itself. Across 4 harnesses, 9 models and 95 tasks, or 3,420 runs, harness scores ranged from 59.8% to 72.0%. In 31.1% of runs the agent finished the task while failing at least one harness criterion, so a passing result can hide an unreliable setup.