Early‑Stopping A/B Tests Inflate False Positives to 28 % – Here’s Why and How to Fix It
A seeded simulation shows that checking an A/B test daily until p < 0.05 pushes the false‑positive rate from the nominal 5 % up to nearly 28 %. The post walks through statistical fixes—pre‑registered stopping rules, alpha‑spending corrections, and Bayesian monitoring—that keep early wins honest.