March 2014 Summaries

The text discusses SingleStore, a distributed database system that allows for high concurrency and scalability. It explains how SingleStore spreads work across a cluster of servers without sacrificing ease of use, using two types of machines: aggregators and leaves. Aggregators act as SQL proxies, coordinating parallel operations and combining results, while leaves store data independently, unaware of the larger cluster. The text also covers various configurations for building a SingleStore cluster, including RAID-like setups with multiple donkeys (servers) and more complex configurations with backup donkeys. It highlights the importance of scaling horizontally over commodity computers, rather than relying on single-box databases, and notes that SingleStore aims to provide a highly-concurrent, distributed SQL database with straightforward programming. The text also touches on details such as table types (reference tables and distributed tables) and aggregator roles (master and children).

The SingleStore database has introduced a new feature to provide detailed statistics about memory use, which was a frequent request from users. The `information_schema` table now exposes this information, allowing users to roll up the data to generate totals such as the size of a table in the cluster. However, skew can occur when rows are sharded according to specific columns, leading to uneven distribution of data across machines. To measure skew, the relative standard deviation is used, with values below 0.05 being considered acceptable. The new feature was developed through collaboration between the "engine" team building the database and the "ops" team building monitoring apps, resulting in a better feedback loop and more efficient software development process. This allows for faster prototyping of visualizations using real data, while also informing improvements to the underlying database features.