Branching Policy Optimization lets LLM agents explore sandbox trees with far lower variance
BPO replaces independent rollout trajectories with a branching tree that shares prefixes, cutting variance and lifting success on WebShop and SWE‑bench by up to 6 points. It halves gradient‑norm variance and hits baseline performance with 38% fewer policy updates, speeding RL training for sandbox‑native LLM agents.