LLMs Spot Spark SQL Bottlenecks, Cutting Debug Time
Expedia’s team fed Spark SQL physical plans to a large language model, which automatically flags anti‑patterns like missing broadcasts, data skew, and excessive shuffles. The LLM then suggests concrete fixes, turning hours‑long manual triage into a few minutes and lowering cluster costs.