Company
Date Published
Author
Luke Sonnet
Word count
961
Language
English
Hacker News points
None

Summary

CUPED, or Controlled Experiment Using Pre-Experiment Data, is a method for improving the speed and accuracy of experimentation by reducing variance through regression adjustment. Utilized within GrowthBook, CUPED allows experimenters to adjust post-exposure data using pre-experiment data, thereby minimizing uncertainty. This approach is particularly effective for metrics with high correlation between pre- and post-exposure data, such as engagement metrics, although it requires reliable historical data for optimal performance. GrowthBook offers customizable settings, including lookback and conversion windows, to tailor CUPED's application according to user behavior and experiment needs. While CUPED enhances experiment efficiency, it necessitates early data collection and does not address the "peeking" problem, which GrowthBook plans to tackle with upcoming sequential testing additions.