10x improved response times, cheaper to operate, and 30% storage reduction: why Instabug chose ClickHouse for APM

Instabug is an SDK that provides a suite of products, mainly crash reporting and application performance monitoring (APM), to empower users to monitor and debug performance issues throughout the mobile app development lifecycle. The challenge of working with performance metrics lies in handling large amounts of frequent events and aggregating them into meaningful insights, requiring visualizations and business logic to extract useful data. Instabug's backend is large-scale, with multiple Kubernetes clusters and dozens of microservices and datastores, handling approximately 2 million requests per minute and terabytes of data daily. The company initially used Elasticsearch for APM but faced performance issues and decided to migrate to ClickHouse, which provided better performance, scalability, and cost savings. The migration process was challenging, requiring careful planning, research, and the development of a versatile infrastructure to support incremental rollout and experimentation. Instabug's experience highlights the importance of thorough research, flexible code and infrastructure, and gradual rollout in migrating to a new datastore.