July 2024 Summaries

ClickHouse has announced its partnership with PeerDB, a Change Data Capture (CDC) provider focused on Postgres. The integration aims to offer seamless and efficient integration between two leading open-source databases: Postgres and ClickHouse. This will enable bridging the gap between transactional and analytical workloads and unlock more value for users and developers. PeerDB offers a fast and reliable change data capture experience, at the scale at which ClickHouse operates, without compromising on speed, reliability, and correctness. The partnership will also expand the connector catalog of source CDC connectors to other types of databases in the future.

ClickHouse, Inc., the company behind the world's fastest real-time analytical database, has acquired PeerDB, a leading provider of change data capture (CDC) solutions for Postgres databases. This acquisition aims to deliver unparalleled real-time analytics and data integration capabilities to customers. PeerDB's technology significantly reduces the time it takes to connect data in transactional databases like Postgres with analytical databases like ClickHouse, boosting developer productivity and time-to-value when building real-time data driven applications. The combined expertise of ClickHouse and PeerDB will pave the way for new features and capabilities aimed at meeting the evolving needs of data-driven enterprises.

Bonree Data Technology migrated from ZooKeeper to ClickHouse Keeper for their observability platform, resulting in significant improvements in performance and resource efficiency. The migration achieved over 75% reduction in CPU and memory usage, with IO overhead decreasing by 8 times and performance improving nearly 8 times. ClickHouse Keeper's compatibility with ZooKeeper client protocols allowed for a smooth transition without server-side modifications. Additionally, the use of C++ in ClickHouse Keeper enhanced overall service stability and improved resource usage efficiency compared to Java-based ZooKeeper.

Modern businesses rely on user-facing dashboards to guide daily decisions, but building these can be time-consuming and resource-intensive. Using an analytical database like ClickHouse and an embedded analytics platform like Luzmo can help overcome these challenges. In this tutorial, we learn how to build a customer-facing dashboard for a property management app using ClickHouse as the data source and Luzmo for interactive visualizations. The process involves connecting your ClickHouse dataset, preparing the data, building the dashboard with charts and interactivity, and finally embedding it into your SaaS or web application. By following these steps, developers can create user-friendly, fast-loading dashboards that integrate seamlessly into their existing applications.

The July ClickHouse newsletter highlights optimal table sorting in the 24.6 release, tracking vessels with ClickHouse and Grafana, and tactics for optimizing CPU usage when running ClickHouse. Featured community member taiyang-li contributes to improving ClickHouse's performance and string processing capabilities. Upcoming events include workshops on migrating from Postgres to ClickHouse and using ClickHouse in Kubernetes. The latest release introduces optimal table sorting, a beta release of chDB for querying Pandas DataFrames directly, and functions for Hilbert Curves. A blog post demonstrates tracking vessels with Python, ClickHouse, and Grafana. Other topics covered include the MergeTree table engine, optimizing ClickHouse for reduced CPU load, and updates to ClickHouse Cloud on Microsoft Azure.

ClickHouse Cloud has introduced enhanced monitoring capabilities with Prometheus integration, making it easier for users to monitor their cloud instances using popular tools such as Grafana and Datadog. This feature addresses a common requirement among organizations adopting cloud services - the ability to monitor uptime and performance independently of the service itself. The implementation ensures that idling instances are not woken up by monitoring requests, allowing them to idle when not in use and minimizing compute costs. ClickHouse Cloud now supports over 1000 metrics for monitoring purposes, with plans to expand this set based on user feedback. New users can receive $300 in credits upon signing up for ClickHouse Cloud.

Get started with ClickHouse Cloud today and receive $300 in credits. To learn more about our volume-based discounts, contact us or visit our pricing page.` This statement is a call to action, encouraging users to start using ClickHouse Cloud while also providing information on how to take advantage of the company's volume-based discount program. The mention of $300 in free credits serves as an incentive for users to sign up and try out the service. By visiting the pricing page or contacting the company, users can learn more about the discounts available and how they can benefit from using ClickHouse Cloud. This statement is written in a friendly and promotional tone, aiming to persuade users to take action and start using the service.

Chase Richards, VP of Engineering at brand protection firm Corsearch, shared his experience with ClickHouse during a recent meetup. The company adopted ClickHouse for its real-time analytics needs, particularly in the areas of anti-piracy and content protection. By replacing MySQL with ClickHouse, Corsearch achieved significant improvements in data compression rates, reduced server management complexity, and enhanced client experiences. Additionally, the team leveraged ClickHouse's SQL syntax to execute vector distance calculations directly, eliminating the need for additional infrastructure or new languages. The adoption of ClickHouse has played a vital role in capturing web traffic data and streamlining complex data pipelines at Corsearch.

In this blog post, we introduced space-filling curves and their application to optimizing queries in a database. We discussed how Hilbert curves can be used to encode multiple columns as a single value while preserving locality, which can improve the performance of range queries and nearest neighbor searches. We also provided an example using NOAA Global Historical Climatology Network data to demonstrate the effectiveness of this technique. We showed that by ordering the table based on Hilbert encoding of latitude and longitude values, we could significantly reduce the number of rows read during a query, resulting in faster performance. This improvement is due to the preservation of spatial locality provided by the Hilbert curve, which ensures that points close in high-dimensional space are part of the same granules. In summary, using space-filling curves like Hilbert encoding can be an effective way to optimize queries for certain types of data, particularly when columns have similar properties and a high number of distinct values. However, it is essential to consider factors such as correlation between columns and the specific requirements of your query before deciding whether this technique is suitable for your use case.

Braintrust is leading a transformation in how AI companies build, test, and improve their products by providing real-time data processing and automated evaluation tools. Founded by Ankur Goyal in 2023, the platform has helped companies like Zapier, Notion, and Airtable gain deeper insights into the performance of their AI models and make faster, data-driven decisions that enhance reliability and quality. Braintrust's integration with ClickHouse enables real-time queries at scale, ensuring immediate data availability and facilitating rapid iteration cycles. The platform uses a multi-database architecture to optimize performance and reliability, with ClickHouse providing the real-time data analytics and high-speed processing necessary for modern AI model development.

Query Insights is a new feature in ClickHouse Cloud that provides users with an intuitive and accessible way to view and interpret their query log data. The system.query_log table contains valuable information about query performance, errors, and time-series metrics, but its complexity can make it difficult for less experienced users to interpret. Query Insights addresses this issue by making the data more interpretable and exposing important top-level query metrics. It also simplifies query debugging and optimization workflows, as well as providing context-based suggestions and insights from the query log. The feature is currently in its first iteration, with future improvements based on user feedback.