May 2022 Summaries

We're excited to invite you to a meet-and-greet event with the ClickHouse team, where you'll get to hear stories of ClickHouse powering high-performance services and explore its speed. The event will take place in person on March 9th, 2023, in Amsterdam, and also virtually on June 8th. This meetup is significant for ClickHouse as an open-source company that values community interaction, and it's a great opportunity to network and learn from the team behind the project. The agenda includes a welcome speech, two keynote talks, lightning talks, and plenty of time for food, beverages, and conversation. The event will be held at the Night venue in Venture Studio, which is hosted by Rabobank. We encourage you to register now to secure your spot and participate in this exciting experience.

In a bid to provide financial services to underserved consumers in Nigeria, QuickCheck, a fast-growing Fintech startup, leverages artificial intelligence and ClickHouse, a column-oriented database, to offer app-based neo-banking products. The company has seen significant success with its mobile app, which has been downloaded by over 2 million people and processed over 4.5 million micro-credit applications. QuickCheck uses ClickHouse for various use cases, including financial data analysis, fraud analysis, and monitoring data, with more than 50 people using dashboards powered by ClickHouse for their daily tasks. The database's performance has been a key factor in the company's decision to migrate its data from Postgres, allowing it to process large amounts of data quickly and efficiently. With ClickHouse's instant query results and column compression feature, QuickCheck is able to create dashboards that were previously impossible to generate due to slow processing times in Postgres. The company's experience with ClickHouse has been overwhelmingly positive, with its team citing the database's performance as "magical" and a key factor in their success.

ClickHouse is being used by DENIC (Deutsche Network Information Center) as a database for their data science platform, which analyzes data from various sources such as relational databases, server logs, and other information sources. Initially, a relational DBMS was used but resulted in too many target tables and containers, making it difficult to administer and overcomplicated. After testing Hadoop and Spark, ClickHouse was chosen due to its efficiency, low administrative effort, and cost-effectiveness. DENIC created a custom data structure for their registry database using ClickHouse's column-oriented databases, which enabled fast queries over large amounts of data. However, they encountered performance issues with the initial filling of the cluster, but were able to optimize the query runtime by creating materialized views and implementing an ARRAY JOIN, resulting in a significant improvement from 5 minutes to about 30 seconds. ClickHouse's performance and expandability have provided DENIC with extensive support in developing their data science platform.

ClickHouse has launched its private preview phase of the ClickHouse Cloud service, inviting users to join and provide feedback on the service before it is opened to the public later this year. The company also released ClickHouse 22.4, which includes new features such as transactions, Keeper load balancing, table metadata caching, Kafka metrics, gap filling, last day of month support for date functions, and H3 geospatial indexing system support. Additionally, ClickHouse has released new documentation pages with a revamped navigation structure and content, including guides on connecting to Grafana, Metabase, Superset, and Tableau, as well as user guides on setting up ClickHouse Keeper and improving query performance using data skipping indexes and sparse primary indexes. Upcoming events include the release webinar for ClickHouse 22.5, ClickHouse Americas Virtual Meetup, and ClickHouse EMEA Virtual Meetup.

Instabug is an SDK that provides a suite of products, mainly crash reporting and application performance monitoring (APM), to empower users to monitor and debug performance issues throughout the mobile app development lifecycle. The challenge of working with performance metrics lies in handling large amounts of frequent events and aggregating them into meaningful insights, requiring visualizations and business logic to extract useful data. Instabug's backend is large-scale, with multiple Kubernetes clusters and dozens of microservices and datastores, handling approximately 2 million requests per minute and terabytes of data daily. The company initially used Elasticsearch for APM but faced performance issues and decided to migrate to ClickHouse, which provided better performance, scalability, and cost savings. The migration process was challenging, requiring careful planning, research, and the development of a versatile infrastructure to support incremental rollout and experimentation. Instabug's experience highlights the importance of thorough research, flexible code and infrastructure, and gradual rollout in migrating to a new datastore.