Home / Companies / ClickHouse / Blog / Post Details
Content Deep Dive

Designing the new async-native ClickHouse Python client

Blog post from ClickHouse

Post Details
Company
Date Published
Author
Introduction #
Word Count
2,717
Language
English
Hacker News Points
-
Summary

clickhouse-connect is the official Python client for ClickHouse, designed to handle high-volume, performance-sensitive workloads. Initially launched as a synchronous client in September 2022, it gained popularity for its robust feature set. Due to demand for asynchronous capabilities, a workaround using a thread pool executor was employed, although it had limitations like thread pool exhaustion and increased memory usage. To address these issues, a native async client was developed, utilizing the "half-sync/half-async" concurrency pattern. This new design separates asynchronous I/O from synchronous processing, using an aiohttp library for high throughput and a bounded queue to manage backpressure, significantly improving query performance and stability under load. Benchmark tests show that the async client offers a 1.16x speedup on average over the legacy client, with more predictable tail latency and efficient resource usage. The async client is now available in version 0.12.0rc1, with feedback actively sought from users to further refine its capabilities.