Starburst scores show near-linear LLM progress since GPT-4o
A small Starburst study finds leading LLMs improving in intelligence almost linearly from GPT-4o in May 2024 through the reasoning era, while still below the most capable humans tested. Starburst is a puzzle game where players infer the laws of physics in a fictional universe; the authors mapped scores to a bespoke 1–5 intelligence scale using limited human results, simulated playthroughs, priors, and intuition. The trend is striking, but the benchmark and human sample are not population-representative.