February 2025 Summaries

The official ClickHouse Tableau Connector has been launched, enabling users to visualize data stored in ClickHouse databases within Tableau Desktop or Server. The connector achieves a 98% compatibility score with Tableau's Data Source Verification Tool and supports various features such as data filtering, mapping, and calculation. Users can download the connector from the Tableau Exchange and follow the provided tutorials and example worksheets to get started. The connector is compatible with ClickHouse Cloud and supports MySQL interface for working with Tableau Cloud. With this integration, users can create interactive dashboards in minutes, visualizing key metrics such as sales figures, user behavior, or property prices. The ClickHouse Tableau Connector enables businesses to unlock actionable insights from their data and make informed decisions.

The ClickHouse database supports over 70 input formats out of the box, but selecting the most efficient format can be overwhelming. To answer this question, a benchmark called FastFormats was developed to measure server-side ingestion speed and hardware efficiency across different input formats. The results show that Native is the fastest and most efficient format, followed by ZSTD for compression. Pre-sorting has a moderate impact, while batching significantly improves efficiency. ClickHouse clients optimize inserts by selecting formats suited to their typical use cases, with those designed for high-throughput inserts automatically choosing Native. The HTTP interface provides greater flexibility than the native interface but lacks some advanced features. The support matrix shows that the C++ and Go clients currently support the native interface, while all main language clients except C++ support the HTTP interface.

This year's FOSDEM conference saw ClickHouse make a strong showing in Brussels, with several team members attending and multiple talks covering topics from powerful new JSON data types to the challenges of fuzzing databases. The conference, one of the biggest open-source events in the world, brought together thousands of developers, engineers, and open-source enthusiasts for knowledge-sharing, project showcases, and deep technical discussions. ClickHouse was well-represented at FOSDEM, with talks covering the new powerful JSON data type, database fuzzing challenges, and the creation of an rDNS map to understand internet structure. The event also included a community dinner where users could meet, discuss database internals, and share ideas over great food and drinks.

ClickHouse BYOC (Bring Your Own Cloud) is a hybrid deployment model that enables organizations to access the real-time analytical capabilities of ClickHouse Cloud while ensuring their data remains entirely within their own AWS Virtual Private Cloud (VPC) environment. This allows customers to leverage the comprehensive security controls of AWS while meeting specific compliance requirements and geographic regulations. By adopting ClickHouse BYOC, customers can benefit from full compute-storage separation, seamless vertical and horizontal scaling of compute nodes, and compute-compute separation, resulting in optimized compute resource usage and lower infrastructure costs. The model provides data security and control, greater operational flexibility, performance predictability, and cloud spend optimization, making it an attractive option for businesses that need to adhere to strict data governance and residency mandates. This is part of a broader collaboration between ClickHouse, Inc. and Amazon Web Services (AWS) to enhance real-time data warehousing, observability, business intelligence, machine learning, and generative AI solutions.

The newsletter discusses various topics related to ClickHouse, a columnar database management system. The launch of JSONBench, a benchmark suite for JSON analytics, is highlighted as a significant development. Ryadh Dahimene introduces the concept of agent-facing analytics and explains how real-time analytics databases can support agentic workflows. Chris Lawrence, Dev Lead at AMP, shares his experience with ClickHouse Cloud and its benefits in transforming data pipelines. The newsletter also features upcoming events, including free training sessions and meetups in various regions. Additionally, it highlights interesting projects using ClickHouse, such as apitally.io and Openpanel, which utilize the database for storing and analyzing data. Finally, video tutorials and a post of the month are featured, showcasing the capabilities and applications of ClickHouse.

ClickHouse is gaining popularity as a bridge between data lakes and data warehouses, offering versatility and flexibility to users. It supports over 60 input and output formats, making it easy to integrate with external systems such as S3, GCS, Azure, PostgreSQL, MySQL, MongoDB, and more. Users can load data from their data lake into ClickHouse for real-time analytics or efficiently ingest data for high-performance analytics. ClickHouse is being used in various scenarios, including frequent queries to data lakes/lakehouses, ad-hoc and federated queries, and data loading from a data lake to ClickHouse. The database is improving its support for open table formats like Iceberg, Delta Lake, and Hudi, with plans to add features such as metadata caching, compactions, and external materialized views. In 2025, ClickHouse aims to enhance the user experience for ad-hoc and frequent queries on data lakes/lakehouses, improve its capabilities for working with data lakes/lakehouses, and introduce an Iceberg CDC Connector in ClickPipes for real-time analytics.

The Postgres CDC connector in ClickPipes is now available in public beta, allowing users to easily replicate their Postgres databases to ClickHouse Cloud with a few clicks. The connector has been integrated natively into ClickHouse Cloud and released after a private preview that received overwhelming customer feedback. Key features include performance optimizations, user-facing alerts for failures or potential issues, a source monitoring page for detailed visibility into replication progress, and open API support for programmatically creating and managing pipes. Additionally, the Postgres CDC connector is built on top of PeerDB's enterprise-ready codebase, which has undergone several improvements to enhance ingestion performance and eliminate reconnections to the replication slot. Pricing will be introduced during the next phase, with the goal of keeping it competitive for real-time analytics use cases at scale.

This is a summary of the ClickPipes Public Beta release of their Postgres CDC (Change Data Capture) connector. The connector allows users to easily replicate their Postgres database to ClickHouse Cloud with just a few clicks, by setting up the service's Data Source tab and selecting the Postgres tile. The connector has been designed with high performance in mind, leveraging features like parallel snapshotting and chunk partitioning to achieve fast data transfer times. Key enhancements include user-facing alert functionality, source monitoring pages, and support for Open API-based programmatic management of pipelines. The connector is built on top of PeerDB's open-source Postgres CDC codebase, with recent improvements including performance boosts and error handling enhancements. Pricing will be introduced in a future GA release, aiming to offer competitive pricing for real-time analysis use cases.

In a rapidly evolving AI landscape, autonomous AI agents are being deployed to interact with real-time analytics databases, marking an interesting shift in how we think about data systems. The emergence of these agents as active users of real-time analytics databases marks both an opportunity and a challenge for the industry. As these agents become more autonomous and their deployment more widespread, new patterns in how they interact with data systems will likely emerge, leading to new optimizations and features. Real-time analytics databases are already being optimized to support high-throughput and exploratory workloads, with features such as improved discoverability, LLM-friendly documentation, scaling for AI workloads, server-side state for AI memory, customized access control models, and a standardized way to connect LLMs with the context they need during specific tasks. The ClickHouse MCP Server is an example of this evolution, providing a bridge between AI-powered applications and data sources, enabling LLMs to list databases on the ClickHouse instance connected, list tables, and run select queries. As organizations continue to deploy AI agents at scale and new use cases emerge, the relationship between agents and real-time databases will likely continue to evolve in ways yet to be anticipated.

ClickHouse version 25.1 has introduced several new features, including a faster parallel hash join algorithm using two-level hash maps, MinMax indices at the table level, improved Merge tables, and auto-increment functionality. The parallel hash join algorithm now uses a single shared two-level hash table to improve efficiency, reducing overhead and improving performance by approximately 36-31% compared to previous versions. The MinMax index feature allows for more efficient querying of columns with sorted data, while the Merge table engine enables combination of multiple tables into a single table using a standardized column structure. Additionally, the auto-increment functionality provides a fast and safe way to generate serial IDs for tables.

ClickHouse バージョン 25.1 は、パラレルハッシュ結合アルゴリズムの速度が高速化され、テーブルレベルの MinMax インデックス機能も改善されたものである。新しいバージョンのリリースでは、2 レベルのハッシュマップを使用してパラレルハッシュ結合アルゴリズムをさらに高速化し、Merge テーブルを改善したことなどが挙げられます。また、バイナリ形式で書き込む前に確認を行う機能や、列名の短縮機能、新しいオートインクリメント関数、Merge テーブルエンジンの改善などが含まれている。

Memorial Sloan Kettering Cancer Center's cBioPortal, a leading platform for precision oncology, has struggled with performance issues due to its reliance on MySQL and old data infrastructure. The team has since turned to ClickHouse, an analytical database that can process massive datasets at high speed, to improve the platform's scalability and speed. With ClickHouse, researchers can analyze complex cancer data faster and accelerate breakthroughs in cancer treatment and personalized care. The implementation of ClickHouse has proven its potential to transform cBioPortal, enabling near-real-time hypothesis testing and accelerating the pace of discovery.

We are excited to announce that ClickHouse Cloud is now eligible for the Microsoft Azure Consumption Commitment (MACC) program, allowing customers to leverage their existing Microsoft Azure commitments towards ClickHouse Cloud usage. This development unifies and streamlines onboarding and billing processes for Azure customers, providing simplified internal procurement, 100% cost coverage with existing budget utilization, and seamless integration into the Azure ecosystem through connectors such as Power BI, while offering a free trial with $300 in credits to experience its full power.