January 2022 Summaries

The new ClickHouse release 22.1 includes several significant features, including schema inference, which automatically detects the data format of a file without requiring manual specification. This feature works for various formats such as Native, Avro, Parquet, and CSVWithNamesAndTypes. Additionally, ClickHouse now supports "schema on demand" queries, allowing users to refine the types in their query with the :: operator. The release also introduces real-time resource usage visualization in clickhouse-client, parallel query processing on replicas, service discovery, sparse encoding for columns, a diagnostic tool for ClickHouse, new integrations, and statistical functions for calculating Cramer's V and Theil's U coefficients. These features enhance the scalability, performance, and user experience of ClickHouse, making it an attractive choice for developers and data analysts.

ClickHouse is an open-source data warehouse solution that helps companies like Admixer, a leading ad-tech company, to handle high volumes of data and provide real-time insights. Admixer was moving from MSSQL to Azure Table Storage to ClickHouse to improve its performance and scalability. The company's Advertising Exchange, which aggregates traffic from external SSPs to place local advertisements on it, required a more robust storage solution. With ClickHouse, Admixer can now handle over 1 million records per second and aggregate data for thousands of publishers. The company chose ClickHouse due to its ability to scale horizontally, provide real-time data processing, and offer cost-effective solutions. ClickHouse architecture has evolved over time, with the addition of new components such as Requests Handler, EventsProcessor, and report generation service. Admixer's experience with ClickHouse has provided valuable lessons for others considering using the solution, including the importance of load testing, choosing the right replication scheme, and monitoring performance.