September 2021 Summaries

The recent release of GrowthBook version 0.5.0 introduces several new features aimed at enhancing data transparency and customization. Users can now export experiment results as Jupyter notebooks, allowing deeper analysis by data teams, with the process reflecting GrowthBook's commitment to transparency by revealing the raw queries and analysis steps. The update also includes customizable per-metric data quality thresholds, enabling users to set parameters for conversion counts and flagging suspicious results based on percentage changes. Additionally, the SDK Dev Mode has been expanded to work in both JavaScript and React environments, facilitating easier testing of experiments by allowing developers to switch between variations on staging builds and development machines. Other improvements include the ability to import running experiments from data sources, a more detailed view of query results, and the availability of the GrowthBook stats engine as a standalone Python package, alongside enhanced documentation and a series of bug fixes addressing issues with forms, emails, and rendering modals.

In August, GrowthBook made significant strides, nearing 1500 GitHub Stars and releasing version 0.4.0 of its open-source A/B testing platform, incorporating user feedback from GitHub and Slack. The new version introduces several key features: per-metric risk thresholds allowing users to balance risk and experiment duration individually for different metrics; experiment-level dimensions enabling more precise analysis based on attributes like "browser" or "traffic source"; and the ability to export UI-created settings to a config.yml file, enhancing configuration management across environments. It also includes enhanced detection of variation ID mismatches for debugging and allows customization of the minimum runtime for past experiments to maintain data integrity.