Beaver benchmark reveals Text‑to‑SQL models stumble on real warehouse quirks
The newly released Beaver benchmark evaluates Text-to-SQL systems on actual enterprise data warehouses, exposing problems like schema rot and idiosyncratic identifiers that synthetic tests miss. Early results show off‑the‑shelf LLMs still falter, highlighting a gap between research claims and production readiness.