Scaffolds can boost AI agent performance up to 100×, dwarfing model differences
The authors analyze the Holistic Agent Leaderboard and find that software scaffolds—tool integrations, memory management, prompting tricks—can change inference cost and accuracy by up to two orders of magnitude. In many cases scaffolds explain more performance variation than the underlying model, reshaping how we evaluate agents and hinting at industry concentration risks.