Company
Date Published
Author
Michael Malis
Word count
1725
Language
English
Hacker News points
None

Summary

Heap, a tool designed for analyzing customer interactions, faces unique challenges in performance engineering due to its ad hoc analysis capabilities, which result in highly complex and unique SQL queries. These queries are executed across a cluster of PostgreSQL instances and are complicated by dynamic schemas that are only fully known at read time. Initially, Heap engineers manually analyzed slow queries using Postgres's EXPLAIN ANALYZE, but this method proved insufficient for handling the vast number of queries. Instead, the company now uses the Postgres extension auto_explain to automatically log detailed query execution data, allowing for efficient analysis and identification of root causes for slow queries. This comprehensive data collection has enabled Heap to prioritize engineering efforts effectively, such as deciding to use table partitioning to enhance query performance, particularly for session and pageview events, which addresses approximately 25% of slow queries. This automated and data-driven approach to query analysis and optimization has significantly improved Heap's ability to manage performance efficiently.