November 2019 Summaries

At Cockroach Labs, expressing gratitude is ingrained in both personal and professional practices, with the concept of "peer acks" serving as a cornerstone of their workplace culture. Initiated in 2016 following a suggestion during an employee survey, peer acknowledgments provided a platform for publicly recognizing colleagues' contributions, initially through emails read aloud by co-founder Ben Darnell at weekly meetings. As the company expanded, the practice evolved with the development of a web app by engineer Andrei Matei, later replaced by a Slack app created by engineering manager Ken Liu, to streamline the process. This tradition not only fosters a supportive work environment but also reflects the company's commitment to appreciation and community, aligning with its open-source ethos. While the methods for delivering acknowledgments have transformed over time, the core intention remains the celebration and gratitude for the efforts of coworkers, embodying a culture of appreciation that extends beyond the workplace.

In a blog post by Tobias Grieger, Cockroach Labs discusses their efforts to enhance CockroachDB's resilience against region failures by implementing Joint Consensus within the Raft consensus algorithm. CockroachDB, a distributed key-value store, relies on consensus algorithms for maintaining data consistency across its numerous independent consensus groups, or Ranges. The post details the challenges faced when altering replication configurations, particularly when moving replicas laterally between nodes to maintain operational efficiency and prevent region-specific failures. Traditionally, these configuration changes risked creating split-brain scenarios due to uncoordinated switchovers among nodes. The introduction of Joint Consensus, which requires agreement from both initial and final configurations, mitigates these risks by preventing split-brain and maintaining high availability even during region outages. This development not only improves CockroachDB's reliability but also contributes back to the broader community, enhancing the etcd/raft library commonly used in systems like Kubernetes. Through this approach, CockroachDB aims to maintain simplicity in external interfacing while addressing complex internal challenges, as demonstrated in their recent 19.2 release.

Cockroach University recently launched with its inaugural course, "Getting Started with CockroachDB," aimed at providing an engaging and structured learning environment for individuals interested in CockroachDB. The course includes hands-on practice, short instructional videos, quizzes, and labs, culminating in a graded final exam that awards a Certificate of Completion to those who pass. Created by Will Cross and Lauren Hirata, the course teaches students how to set up an insecure cluster, understand data distribution and replication, and grasp basic database guarantees. To further support learning, a Slack channel has been established for student interaction and staff assistance during designated office hours. The initiative marks the beginning of Cockroach University's offerings, with plans already underway for a more advanced course focused on application development with CockroachDB.

The analysis by Peter Mattis compares CockroachDB and Yugabyte, focusing on competitive benchmark claims and highlighting significant architectural and performance differences. CockroachDB, a distributed SQL database built from scratch for distributed environments, generally outperforms Yugabyte across various benchmarks, attributed largely to its range partitioning and efficient transaction handling. In contrast, Yugabyte, which integrates PostgreSQL with its distributed KV store, uses hash partitioning by default, leading to performance issues in range scans. The study reveals Yugabyte's limitations with schema changes and transaction retries, and it highlights CockroachDB's superior handling of distributed transactions and schema changes. The discussion underscores the complexities of benchmarking and the impact of architectural choices on database performance.

In a comparative analysis of CockroachDB and Amazon Aurora using the industry-standard TPC-C benchmark, CockroachDB demonstrated significant scalability and performance, processing 100 times the throughput of Aurora's last published metrics. CockroachDB's recent achievement of passing TPC-C at 100,000 warehouses with a max throughput of 1.2 million transactions per minute (tpmC) highlights its ability to maintain serializable isolation while scaling, thus protecting data integrity. The TPC-C benchmark, which simulates e-commerce or retail operations, involves a complex mix of transactions and remains a critical measure of OLTP workload performance. CockroachDB's architecture allows it to scale more effectively than Aurora, supporting 50 billion rows and over 8 terabytes of frequently accessed data. Cockroach Labs emphasizes the reproducibility of their benchmarks, providing detailed steps for others to replicate their results, and offers a dedicated performance page with the latest testing information.

CockroachDB 19.2 has been released, enhancing the latency, reliability, and usability of the distributed SQL database while introducing Cockroach University, a free online learning platform. The update includes significant improvements such as reduced latency in distributed transactions through a new atomic commit protocol called Parallel Commits, which halves transaction latency by minimizing consensus rounds. Additionally, vectorized execution is enabled by default, boosting analytical query performance. The release also enhances ease-of-use for distributed clusters, allowing precise control over data backup locations to reduce costs and comply with regulations like GDPR. New features simplify geo-distributed database configuration, including more accessible zone configurations and improved partitioning table views. The announcement encourages users to explore detailed release notes, participate in a technical demo, engage on a new Slack channel, and watch an informative webinar.

CockroachDB's new "Parallel Commits" protocol, introduced in version 19.2, aims to significantly reduce the latency of distributed transactions by allowing them to be committed in a single round-trip of distributed consensus, thereby improving performance in globally distributed deployments. This approach overcomes the limitations of the traditional two-phase commit protocol, which was prone to increased latency due to the need for sequential consensus rounds. Parallel Commits achieve their goal by changing the transaction commit condition and introducing a "STAGING" state, which allows transactions to be considered committed once all write intents have been successfully replicated, even if they haven't all been confirmed individually. The protocol incorporates ideas from various academic solutions and unifies distributed transactions and consensus under a single framework using Flexible Paxos, ensuring that transactions maintain atomicity and durability. Extensive benchmarks, including TPC-C tests, have demonstrated that Parallel Commits halve the latency of cross-range transactions, making CockroachDB more efficient, especially when tables have multiple secondary indexes or when deployed in geo-replicated clusters.

ESCAPE/19, the inaugural Multi-Cloud Conference held in New York City, highlighted the growing acceptance and inevitability of multi-cloud environments in today's technological landscape. Keynote speakers and panelists, including Armon Dadger and Kris Nova, discussed the complexities of navigating multi-cloud storage, networking, and security challenges, emphasizing the need for agnostic tools for seamless integration across different cloud platforms. The conference also addressed the environmental impact of cloud consumption, with a call for increased reliance on renewable energy to reduce carbon emissions. Ethan Jackson highlighted the difficulties faced by developers in adapting to the complexities of microservices, while Zac Smith underscored the political implications of data localization regulations affecting enterprises globally. The concept of "accidental multi-cloud" was also explored, noting how mergers and acquisitions often result in unplanned multi-cloud scenarios, compelling companies to adapt to diverse cloud platforms. Overall, ESCAPE/19 underscored the importance of collaboration, innovation, and strategic planning in achieving efficient and sustainable multi-cloud operations.