March 2016 Summaries

Cockroach Labs has launched the beta version of CockroachDB, an open-source database designed to address the evolving needs of businesses by offering scalability, survivability, and SQL compatibility, all while ensuring strong consistency. The development team has grown significantly, and the project has matured rapidly, with the inclusion of SQL being a pivotal decision to avoid leaving developers with incomplete functionality. The database aims to support real-world use cases by allowing for distributed transactions, online schema changes without downtime, self-healing capabilities, and an intuitive deployment process. As the system continues to develop, future enhancements will focus on expanding SQL capabilities, improving stability, and refining performance to establish CockroachDB as a scalable database solution comparable to established platforms like Postgres and MySQL. The CockroachDB beta invites new users to test its features, participate in its community, and contribute to its ongoing evolution.

Managing notifications from busy GitHub repositories can be overwhelming, especially when monitoring active projects like CockroachDB, which averages 81 pull requests and 440 comments weekly. The GitHub watching feature is criticized for its lack of granularity, prompting the development of tools like repo-digest, a GitHub pull request digester designed to condense PR activity into a daily, manageable format. By utilizing the GitHub API, repo-digest retrieves and organizes PRs from specified repositories, sorting them by total changes to emphasize significant updates. Users can customize the time frame and repositories to be monitored, and the output can be styled using templates for email distribution. The tool also highlights key subdirectories affected by changes, providing a deeper understanding of the focus and impact of each pull request.

As CockroachDB nears its beta release, documenting its SQL implementation has become crucial, particularly through the use of intuitive grammar diagrams. Inspired by SQLite’s effective use of railroad diagrams, the author sought to create similar visuals for CockroachDB using a combination of yacc, EBNF, and various conversion tools, such as the Railroad Diagram Generator. The process involved converting the existing SQL grammar from yacc files into EBNF format, refining the diagrams for usability by inlining and simplifying complex statements, and filtering out unimplemented SQL expressions to maintain accuracy. This approach allows for the automatic generation of SQL diagrams, which are easily navigable and updateable, enhancing the overall documentation and user experience.

CockroachDB, a distributed SQL database, encountered a significant performance issue during its testing phase, which involved a rapid decline in operations per second when writing random data into a table. This deterioration, observed by engineer Matt Tracy, was traced back to the inefficiency in handling deletion tombstones in RocksDB, the database's storage engine. The issue arose when the system attempted to iterate through a large number of deleted keys, which slowed down the process significantly. To address this, the team implemented a solution utilizing RocksDB's ability to set an upper-bound limit for iterator operations, effectively curtailing unnecessary iterations through deleted keys. This fix not only stabilized performance but also enhanced the database's overall top-end efficiency, leading to substantial improvements in various benchmarks.