Company
Date Published
Author
Tony Burke, SolarWinds
Word count
1189
Language
English
Hacker News points
None

Summary

Tony Burke, a seasoned software engineer, joined SolarWinds in 2023 with nearly three decades of experience solving complex technical issues and architecting solutions for cloud and SaaS companies. At SolarWinds, he faced unique challenges in managing the scale and high-stakes nature of its data operations, processing 3 million telemetry messages per second, averaging 550 megabytes per second, and peaking at bursts of a gigabyte. To tackle these challenges, Tony turned to ClickHouse, fine-tuning their system and optimizing queries for time-sensitive metrics, building a data platform capable of scaling with speed and precision. He identified key areas for optimization, such as reducing index granularity, which led to significant improvements in query performance, including a 60% reduction in query times. However, this came with tradeoffs, including increased memory usage and merge times. To balance gains and tradeoffs, Tony also optimized filesystem reads by switching from the default `pread_threadpool` setting to `pread`. SolarWinds adopted the Bring Your Own Cloud (BYOC) deployment model on AWS to streamline management of its ClickHouse deployments, ensuring greater control over security, network configurations, and data compliance. By leveraging ClickHouse's shared-storage architecture with Amazon S3 for durable storage, SolarWinds can independently scale compute resources, resulting in infrastructure cost savings and simplified maintenance. Ultimately, ClickHouse is the foundation of SolarWinds' real-time observability platform, empowering IT teams to act with speed, precision, and confidence.