Shipping LLM features is mostly boring reliability engineering, not model engineering
Once an LLM feature reaches production, the model is the easy part. The real work is unglamorous scaffolding: timeouts, retries, validation, fallbacks, observability. Datadog data shows about 5% of LLM requests fail in production, with most killed by rate limits rather than bad prompts. Users can't tell a wrong answer from no answer, but they only forgive one.