Home / Companies / Statsig / Blog / Post Details
Content Deep Dive

Debugging sample ratio mismatch: Custom dimensions in Statsig

Blog post from Statsig

Post Details
Company
Date Published
Author
Daniel West
Word Count
358
Language
English
Hacker News Points
-
Summary

Statsig, a platform used by companies like Vista for running experiments, has introduced new debugging capabilities to address Sample Ratio Mismatch (SRM) issues that can lead to skewed and unreliable experiment results. SRM occurs when there's an uneven distribution of users between control and test groups, often due to technical issues like website crashes. Previously, Statsig's debugging tools were limited to a preset list of dimensions, but the recent update allows customers to define custom dimensions for more precise analysis. By enabling the analysis of these custom dimensions, Statsig offers deeper insights into the root causes of SRM, helping customers with complex setups to diagnose and resolve these issues effectively, ensuring the validity of their experiment findings.