LLMs Collapse on High‑Dimensional Tables, Explaining Their Lag Behind XGBoost
Feeding a frontier LLM a single prompt with full training and test data, the authors eliminated noise, CSV formatting, tokenisation, and batch size as failure causes. Only dimensionality mattered: accuracy dropped as features grew, while nine classical baselines stayed flat or improved. This accounts for LLMs’ consistent lag behind XGBoost on tabular tasks.