BIRD benchmark’s gold queries are buggy, inflating text‑to‑SQL scores
sqlsure discovered that the BIRD (and Spider) text-to-SQL benchmarks contain gold queries that are outright wrong. A deterministic checker flagged 15 dev queries, confirming 14 errors, including a query that overcounts by a factor of eight. This means model accuracy scores have been inflated, misleading both research and enterprise deployments.