Lessons From Hookdeck’s Quest for Scalability

Hookdeck recently faced scalability challenges due to an increase in the number of daily events being ingested by new customers. The team tackled these issues by focusing on performance optimization and operating Hookdeck at scale. They encountered two main problems: a database timeout error and out-of-memory errors on some worker pods in their Kubernetes cluster. By optimizing their PostgreSQL database queries, they resolved the timeout errors, while addressing memory utilization issues through adjustments to Google Pub/Sub's settings and using profiling tools. The team emphasizes the importance of monitoring resource utilization and paying close attention to I/O operations when tackling performance optimization.