London Meetup Report: How Cloudflare Processes Hundreds of Millions of Rows per Second with ClickHouse

ClickHouse is a popular open-source column-store database that has been adopted by Cloudflare, one of its earliest adopters. Cloudflare uses ClickHouse for various use cases, including HTTP and DNS analytics, logging analytics, traffic management, cloud worker runtime analysis, internal analytics workload, customer dashboards, firewall analytics, and radar. The company's massive scale of operations has led to the deployment of over 1,000 active replicas, processing hundreds of millions of rows every second. Cloudflare has leveraged ClickHouse's powerful sampling features for their diverse datasets and has overcome challenges such as memory consumption by shifting from the 'hashed' layout to the 'hashed array' layout. The company has also developed solutions to address these issues, including the 'ClickHouse Dictionary Dumper', which separates data production from consumption. Cloudflare continues to evolve its use of ClickHouse, with potential enhancements including in-memory compression and dynamic dictionaries.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.