January 2024 Summaries

GrowthBook has released version 2.7, featuring several enhancements such as sticky bucketing, reusable targeting conditions, an experiment health tab, and optimizations for fact table queries. Sticky bucketing ensures consistent user experiences by remembering the first variation a user sees, even if conditions change, and is currently supported in the latest JavaScript and React SDKs. Reusable targeting conditions allow for more complex and versatile Condition Groups, which can be applied across multiple experiments and features. The experiment health tab centralizes data quality checks, enabling automatic dimension checks and easier detection of issues like Sample Ratio Mismatch. Fact table query optimization reduces query numbers for experiments with multiple metrics, potentially lowering costs for data warehouses like BigQuery and Snowflake, and is available to Enterprise customers. Additionally, GrowthBook introduces a "Make Changes" button to guide users through safely updating live experiments without introducing bias or errors, with additional release strategies for Pro and Enterprise users. A new best practices guide and a contextual AI bot have also been added to assist users in optimizing their use of GrowthBook, along with various other improvements and bug fixes.

In GrowthBook 2.7, making changes to running experiments has become safer and more flexible, thanks to guided flows and sticky bucketing, which help avoid bias and preserve user experience. When decreasing traffic to an experiment, users have options to either start a new phase, apply changes to everyone, or apply changes only to new traffic, with sticky bucketing enabling the latter to maintain data integrity and avoid bias. Increasing traffic is less complex and can be done without starting a new phase, by adjusting targeting or traffic percentages. Restarting experiments may be necessary due to bugs or design changes, and GrowthBook provides comprehensive guidance to ensure informed decision-making, as well as detailed documentation on sticky bucketing and experiment modifications.

In January 2024, GrowthBook 2.7 introduced significant SQL performance enhancements for Enterprise users by optimizing the way metrics are processed, building upon the Fact Tables feature added in version 2.5. These enhancements allow for shared SQL definitions across related metrics, minimizing duplicate work and reducing costs, particularly for large companies using data warehouses like BigQuery or Snowflake. With a new method that combines multiple metrics from the same Fact Table into a single query, the need for temporary tables is eliminated, enabling substantial performance gains. The system now efficiently handles complex experiment queries with potentially hundreds of columns, using CASE WHEN statements to accommodate different filters for each metric without interference. Additionally, advanced functionalities like Ratio Metrics and CUPED are now more cost-effective, facilitating a broader application without performance concerns. By transitioning from numerous narrow queries to fewer wide ones, GrowthBook achieves up to a 90% cost reduction, and potentially more with advanced metrics, thereby encouraging users to conduct more experiments and drive success through increased testing velocity.