LLM Test‑Time Sampling Clusters into Basins, Undermining Majority‑Vote Accuracy
The ARBITER study finds that test‑time sampling in LLMs produces reasoning trajectories that collapse into a few ‘basins’, causing majority‑vote mechanisms to pick the most stable but often incorrect answer. By modeling basin interactions using only the model’s own samples and hidden states, ARBITER’s additive evidence recovers up to 22% of the oracle headroom on math benchmarks.