How to plan test duration when using CUPED

CUPED (Controlled Utilization of Pre-Existing Data) is a statistical technique designed to enhance the sensitivity of controlled experiments by reducing variance in key performance indicators (KPIs), allowing for shorter test durations or lower sample sizes while maintaining statistical power and minimum detectable effect (MDE). The method leverages pre-existing data to adjust for variability unrelated to the experimental treatment, thereby isolating the true effect of the treatment. By incorporating CUPED into the planning phase of tests, experimenters can significantly reduce the sample size required by calculating the Pearson correlation between historical and experimental data, and adjusting accordingly. This technique is particularly beneficial in A/B testing scenarios where detecting small differences between groups is crucial. With its ease of implementation and ability to optimize resource use, CUPED is widely adopted in A/B testing platforms like Eppo and Statsig, making it an invaluable tool for data scientists and analysts aiming for efficient and reliable experimental designs.