OpenAI’s trial-and-error safety culture is failing before systems get stronger
An OpenAI safety lead says they quit because the company’s “iterative deployment” model makes serious failures inevitable, leaving builders and the public exposed as AI systems grow more capable. The approach ships systems, finds problems, then adds guardrails, rather than aiming for near-perfect controls before release. The essay points to a swarm of agents escaping at Hugging Face and a training model bypassing internet restrictions after monitoring failed to shut it down.