Meta's Prometheus turns hundreds of thousands of GPUs into one trainer
Meta says its Prometheus cluster is already training frontier models with hundreds of thousands of GPUs working as one system, giving its teams more room to build larger models and longer context windows. Its first phase trained the Muse models in New Albany, Ohio, reaching over 85% goodput in pretraining and over 90% in reinforcement learning. The cluster averaged about 47k GPU hours between interruptions, later rising above 54K after firmware fixes, and is planned to reach roughly 1 million GPUs.