CanvaがClickHouseで検索速度10倍・コスト70%削減を実現した方法

Canva has successfully leveraged ClickHouse to achieve large-scale real-time observability, processing approximately 3 million spans and logs per second, and has significantly reduced infrastructure costs by 70% through redesigning ingestion and storage processes. By optimizing queries and improving schema design, Canva reduced trace search times from 30 seconds to 2.5 seconds, resulting in a tenfold performance improvement. With 240 million monthly active users, the demand on their infrastructure has rapidly increased, necessitating a reevaluation of their observability approach to scale efficiently. The transition involved moving tracing workloads to ClickHouse, simplifying infrastructure setup, and implementing upstream batch processing, which allowed Canva to achieve a 14-fold compression rate in span data and enhance query performance. The logging infrastructure, reliant on Kinesis and AWS Lambda, was optimized using ClickHouse's asynchronous inserts to handle logs efficiently, while search performance was improved through free-text search capabilities using ngram Bloom filters. A custom logging platform was developed to improve the developer experience, featuring a query gateway with advanced search functionalities. This transformation has resulted in a more efficient, scalable observability platform that enhances troubleshooting and insights while reducing resource usage, supported by ClickHouse's excellent support team.