Relay: self-hosted LLM gateway that auto-routes models via eval-gated routing
Relay is a single-binary Go gateway that presents OpenAI and Anthropic endpoints, then routes each request to the cheapest or highest-quality model based on an on-board evaluation harness. It works with any API key, offers failover, caching, and zero telemetry, letting teams control cost and privacy.