Home / Companies / Statsig / Blog / Post Details
Content Deep Dive

Calculate exact relative metric deltas with Fieller intervals

Blog post from Statsig

Post Details
Company
Date Published
Author
Liz Obermaier
Word Count
1,076
Language
English
Hacker News Points
-
Summary

Fieller's method, now available in Statsig, offers a more precise way to calculate confidence intervals for relative metric deltas compared to the Delta Method, which uses an approximation for variance. This approach is particularly useful when dealing with noisy data or small denominators, providing an exact solution for confidence intervals, which are typically asymmetrical, unlike the symmetrical intervals from the Delta Method. Fieller intervals are recommended for new experiments as they offer more accurate and reliable results, especially in edge cases where the control group's mean isn't statistically distinguishable from zero, potentially leading to unbounded confidence intervals. While existing Statsig users may notice changes in notation and reporting style due to the switch to Fieller intervals, the increased precision justifies these adjustments, and the platform encourages adopting this methodology for more dependable experimental analysis.