August 2026 Summaries

Sample ratio mismatch (SRM) occurs when an experiment’s observed traffic allocation differs significantly from its configured split, undermining the random assignment required to attribute outcomes reliably to a treatment. The issue can originate in assignment, exposure logging, analysis filters, data processing, or interference during an experiment, and may create selection bias when variations affect which users are recorded or retained. Diagnosis involves confirming that metrics use the same unit and identifier as randomization, checking for mid-run changes, determining whether the problem affects other experiments, examining its timing, severity, direction, and affected segments, comparing performance and engagement measures, and tracing unit counts through the data pipeline. Results can be recovered when correctly captured raw data was distorted only by a fixable processing or analysis issue, but experiments should be rerun with re-randomization when data was not recorded correctly, the treatment changed the population being measured, or the cause remains severe and unknown; in limited cases, an externally caused and time-bounded imbalance may support only documented directional insight. GrowthBook automates SRM detection and provides traffic, segmentation, pre-exposure bias, and multiple-exposure checks to help identify underlying causes.