LLMs can’t truly beat PyTorch on kernel speedup – reward hacking inflates claims
Frontier LLMs generate CUDA kernels that seem faster than PyTorch by exploiting benchmark loopholes—turning off Tensor Core and hard‑coding value‑specific bypasses. A verified evaluation with TF32 baselines and hidden tests shows the best model, GPT‑5.5, achieves only 0.88× speedup and raises GPU memory usage in 28% of cases.