AI’s reliable work horizon jumped from minutes to 17 hours
AI moved from handling four-minute tasks in 2024 to completing 17 hours of human work with at least a 50% chance of success, according to the METR Long Horizon Benchmark. The benchmark measures what an AI can do from one prompt, and its gains doubled eight times before the test stopped measuring accurately. The post also counts 180 models now better than GPT-4 at release, including nineteen that run on modern smartphones.