Introducing group.experiment(): Safely test changes to production code
Blog post from Inngest
Lauren Craigie's blog article introduces a new SDK primitive called group.experiment() by Inngest, designed to enable safe testing of changes to production code by comparing different variants on live traffic using execution data. This tool allows developers to measure and analyze the performance of various implementations, such as workflow steps or data transformations, by leveraging execution data like timing, status, and retries, which are recorded for every run. Unlike traditional experimentation tools that rely on input and output data, group.experiment() is integrated directly into the execution layer, allowing for a more comprehensive comparison of variants based on real production metrics such as speed, reliability, and cost-effectiveness. This approach eliminates the need for rewrites or pre-wired data pipelines, offering a more efficient and accurate method for determining the best variant. The functionality supports multiple selection strategies, including weighted random splits, bucket-based assignments, custom logic, and fixed selections, providing flexibility in experimentation and deployment scenarios. The tool is available in the Inngest TypeScript SDK 4.8.0 and later, encouraging users to upgrade to take advantage of its features.
No tracked trend matches for this post yet.