Quail pushes AI-SQL past a billion tokens per minute on one H100
Quail lets AI-SQL workloads process over a billion tokens per minute on one H100 GPU, bringing the cost below 6¢ per billion tokens on Modal. It plans database-style queries before sending their many small model calls to the inference engine, instead of treating each call like an arbitrary user request. On Modal’s AI-SQL benchmark, it runs 1.84x faster than the vLLM baseline across tasks, and the authors provide a command to try it now.