トラブルは向こうからやってくる: Cloudflareがクアドリリオン(千兆)行規模の分析をスケールするためにClickHouseをどう活用しているか

Cloudflare has prioritized building a resilient infrastructure capable of withstanding inevitable failures and surges in traffic, serving about one-fifth of the world's websites. The company has been operating the open-source version of ClickHouse for nearly a decade, showcasing its exceptional query performance by scanning vast amounts of data in under two seconds even during simulated outages. ClickHouse's appeal lies in its node design that reduces negotiation needs, simple HTTP integration, and minimal scaling adjustments, making it suitable for Cloudflare's extensive operations. Jamie Herre, Cloudflare's Senior Director of Engineering, emphasizes that scaling is an ongoing journey of adaptation to more data, complexity, and failures, rather than a finite process. Demonstrations have proven the system's resilience and responsiveness, maintaining performance stability under extreme conditions without sacrificing speed. ClickHouse's features, like "soft cluster" capabilities and SQL dialect, offer flexibility and efficiency, while its open-source community provides valuable contributions and insights. Herre advises companies to prepare for scaling challenges proactively, as disruptions are inevitable, and the key is readiness when they occur.