November 2018 Summaries

The MergeTree cloud tables system is designed to provide data locality and scalability for large datasets, with features such as automatic sharding, replication, and rebalancing of data across servers in a cluster. The system uses a sharding key to distribute data across servers, allowing for efficient querying and retrieval of data. The cloud table creation process involves specifying the sharding key, which is used to create multiple virtual shards that are then mapped onto real servers. The number of virtual shards can be adjusted dynamically based on the amount of data in the table. The system also allows for parallel insertion of data into multiple shards, reducing the load on individual servers and improving overall performance. However, issues such as handling large amounts of data, maintaining data locality, and ensuring atomicity of inserts and updates need to be addressed through various techniques, including intermediate mappings, splitting each shard into smaller parts, or using a spreading factor to distribute data evenly across servers.

ClickHouse is a popular open-source database management system that has gained significant attention in recent years due to its ability to provide subsecond latencies for analytics. The Percona Live conference featured three talks on ClickHouse, with only one coming from Yandex, and offered a chance for attendees to engage with the community at the Altinity-sponsored booth. Alexander Zaitsev, CTO of Altinity, provided an overview of ClickHouse along with case studies and best practices, while Aleksey Milovidov, lead developer from Yandex, discussed unique features of ClickHouse in his presentation. Additionally, data engineers from MessageBird showcased their use of ClickHouse for analyzing message delivery processes, highlighting the system's flexibility and capabilities.

The 20th ClickHouse Meetup was held in Amsterdam, attracting attendees from across Europe and several local companies. Data engineers from MessageBird showcased their use of ClickHouse for analyzing message delivery processes, while representatives from the ClickHouse team demonstrated recent string processing optimization features. A benchmark by Altinity compared ClickHouse to TimescaleDB and InfluxDB under time-series workload, highlighting its performance advantages. Meanwhile, Crobox presented a Scala reactive client with a built-in DSL for query building, and Yandex shared insights on performance analysis of ClickHouse queries.

ClickHouse has gained significant interest among Chinese experts, with a rapidly growing number of attendees at the recent Beijing meetup, which was fully booked despite only 170 people being able to attend in person. Many Chinese companies have already adopted ClickHouse in production and are willing to share their experiences. Notable speakers from top tech companies such as Yandex and Tencent presented on various topics, including string processing optimization and how ClickHouse fits into their data processing infrastructure. The event also highlighted the contributions of Chinese developers to the ClickHouse source code, with several individuals receiving awards for their outstanding contributions. The ceremony recognized the achievements of Winter Zhang, Amos Bird, Sundy Li, and William Kwok, who were awarded for their development work on ClickHouse features and improvements. The special award was given to William Kwok for his role in organizing the Chinese ClickHouse community and meetups.

ClickHouse is a popular open-source database management system used for analyzing large amounts of data, particularly in the Big Data industry. The company's participation in the Analysys A10 conference in Beijing showcased its capabilities and popularity among top management, analysts, and IT professionals from various Chinese companies. The team won the OLAP contest for the second consecutive year by utilizing ClickHouse as the core of their solution to calculate complex marketing funnels efficiently. This achievement highlights ClickHouse's ability to handle large data sets and provide fast performance results, making it an attractive choice for businesses in the Big Data industry.