LoRA Makes Async GRPO Practical Across Separate HF Jobs
Distributed RL training usually hits a networking wall when the trainer and inference workers cannot share a node. This Hugging Face setup replaces NCCL with a Storage Bucket and a lightweight proxy, sending megabyte-scale LoRA adapters instead of repeatedly moving a roughly 3 GB model. Across five runs, the same 500-step recipe fell from 3 hours 27 minutes to 53 minutes, while routing rollouts by KV-prefix locality keeps inference from wasting its cache.