September 2021 Summaries

Heap, a product analytics platform, encountered a significant operational incident while attempting to roll out over one million indexes across its distributed Postgres cluster, which led to increased ingestion latency—delays in data visibility for users. The incident occurred when the creation of a new index type for their Effort Analysis feature inadvertently triggered the creation of additional core indexes, straining the system beyond its capacity. Despite precautions like scheduling and small-scale testing, the unexpected system stress highlighted deficiencies in their risk assessment approach, particularly the need for more comprehensive monitoring of index sync queue sizes and the impact of ambient syncs. The incident underscored critical lessons for managing large, distributed systems at scale, such as the importance of aligning testing conditions with real-world scenarios and understanding system load changes over time. As Heap continues to grow, it aims to refine its process by marking certain indexes for "backfill only" to prevent similar issues and to continuously adapt its practices to accommodate the evolving demands of its infrastructure.

The company transitioned from an annual Hack Week, focused on creating proof-of-concept demos, to a biannual Ship Week, emphasizing the delivery of live, ready-to-use features within a week. This change aims to address the limitations of Hack Week, where promising ideas often remained unrealized due to the time required to develop a full version. Ship Week has proven successful, generating ambitious projects that bring immediate value and revealing gaps in the regular engineering processes, such as prioritization and execution inefficiencies. It also serves as a diagnostic tool, highlighting areas that may lack ownership or are underexplored, and showcases the cost of adherence to typical processes. Despite its success, challenges remain, such as post-week project wrap-ups and the unique burden on platform teams during Ship Week. The company continues to iterate on the event to enhance its effectiveness, encouraging collaboration and early design involvement to maintain project momentum.

In this guest article on Fairygodboss, the author shares their experiences and insights from taking parental leave from their role as EVP of People at Heap, emphasizing the company's supportive policies for new parents. Since returning, the author has witnessed significant changes at Heap, including a new CEO and an influx of over 50 new team members, while appreciating the company's generous four-month maternity leave, flexible virtual-first working model, and a stipend for expenses like food delivery and childcare. The author provides advice for working parents preparing for parental leave, such as preparing an offboarding plan, deciding on communication preferences during leave, intentionally reboarding upon return, and maintaining open communication with managers. The article underscores the importance of supportive workplace policies and offers practical tips for parents navigating parental leave and reentry into the workforce.

A data scientist at Heap describes the development and impact of the Step Suggestions feature, part of the Heap Illuminate suite, which automates the creation of multi-step funnels for product analytics. This feature leverages statistical principles to suggest intermediate steps in user flows, helping businesses better understand where users drop off and prioritize improvements. By codifying criteria for effective funnel steps—ubiquity and divisiveness—the feature enhances the accuracy of funnel analysis, reducing reliance on the traditional "guess and check" method. The data scientist highlights the feature's success, noting that 38% of funnels receive useful suggestions, with many steps proving both ubiquitous and divisive, thereby offering actionable insights for product optimization. This automation not only speeds up the analytics process but also improves accuracy by avoiding common pitfalls such as adding unnecessary steps that users can skip. The article emphasizes the potential of high-throughput data science in solving numerous analogous problems across thousands of companies, showcasing the broader applicability and value of Heap's analytics tools.

Heap's initiative to integrate a component library into its web app aims to streamline development and enhance consistency across its growing engineering teams. This effort emerged as multiple teams began handling front-end tasks, necessitating a uniform approach to component design and implementation. By collaborating with the Design team, which spearheads the visual aspects through a UI kit in Figma, a 1:1 correspondence between UI elements and library components is being pursued. The library's creation involves establishing contribution guidelines focused on code isolation, comprehensive examples, robust documentation, and stringent testing, all facilitated through tools like Storybook and Chromatic. Initial challenges included selecting components that met isolation criteria, leading to the migration of simpler components like RadioButton before tackling more complex ones. Achieving cross-team buy-in was vital, accomplished by involving all engineering teams in guideline reviews, cross-team code reviews, and shared responsibilities in moving components to the library. The project has led to a functional component library with a live Storybook, and ongoing efforts include refactoring existing components and adding new ones during project planning, fostering a more cohesive design system that supports Heap’s analytics platform.