A 99% churn score can hide leaky data and bad business decisions
The IBM Telco churn benchmark can report 99.01% accuracy while producing misleading model comparisons and costly retention decisions. The audit finds that pre-split SMOTE, a synthetic-data technique, boosts churn F1 by 13.1 points, with about 36% of synthetic training examples interpolating test instances. Under the paper’s $50 offer and 24-month customer-value proxy, the cost-optimal threshold saves about $77,000 per 1,000 customers versus an F1-focused approach.