Taste-Bench finds frontier agents right at only 59.7% of forks
Taste-Bench tests whether an LLM agent can choose the better next step when a long task splits, a capability that determines whether it burns its remaining budget on a plausible mistake. It gives the model the task, the path so far, and two options, then checks the choice against what happened later; the benchmark contains 502 questions mined from software engineering and machine-learning trajectories. The best listed score is 59.7%, and a choice counts only when the model gets it right in both option orders.