Google’s Ironwood Makes TPU Inference a Third-Party Business
Google’s TPU advantage used to be mostly trapped inside Google. Third-party InferenceX tests now put TPUv7 Ironwood at up to 50% better performance per dollar than Nvidia’s B200 and B300, across much of the performance-cost curve. The harder part is no longer the silicon alone: Google must finish externalizing the TorchTPU serving stack, including speculative decoding, disaggregated prefill, and KV-cache management, before customers can capture that advantage reliably.