Model routing costs more than token prices: why cheap LLMs win in practice
IBM Research shows that real‑world model routing is a systems optimization nightmare, not a simple classification task. Hidden costs like cache pricing, invisible task difficulty, and infrastructure latency can flip the expected cheap‑model‑wins on their head, forcing routers to juggle cost, quality, compliance, and speed simultaneously.