March 2019 Summaries

YugabyteDB 1.2 has passed Jepsen testing, a comprehensive benchmarking framework for distributed systems, using the YCQL API. The test suite was analyzed by Kyle Kingsbury, the creator of the Jepsen test suite, who identified three safety issues that were fixed in YugabyteDB 1.2. These issues related to snapshot isolation, linearizable counters, and sets, which were addressed through changes to the locking subsystem, Raft implementation, and HLCs. The testing also highlighted the importance of clock skew tolerance, high-performance multi-region transactions, and stable infrastructure. However, the testing revealed some areas for improvement, including new user experience, partition tolerance, and documentation. Despite these challenges, YugabyteDB demonstrated its ability to handle massive scale and low latencies, making it a promising database solution for distributed systems.

The development of Yugabyte SQL (YSQL), a distributed, highly resilient, PostgreSQL-compatible SQL API layer powered by DocDB, has been an engineering achievement that brings together two iconic database technologies. The YSQL architecture is designed to be scalable horizontally, with no single point of failure, and provides high resilience with native failover and repair capabilities. It also enables geo-distribution across multiple regions and supports cloud-native operations with zero downtime for schema changes or infrastructure migrations. Through a series of posts, the authors explain their design principles, tradeoffs, implementation details, and lessons learned, highlighting the benefits of YSQL as a PostgreSQL-compatible database that can complement existing investments in YugabyteDB.

YugabyteDB is inspired by the Google Spanner architecture, which was designed to power geo-distributed, random access OLTP workloads. The database's logical architecture consists of two layers: the upper Yugabyte Query Layer (YQL) and the lower DocDB document store. DocDB is a high-performance distributed document store that offers strong consistency semantics, including single-row linearizability and multi-row ACID transactions. It uses Raft consensus algorithm to ensure fault-tolerant data replication across multiple nodes. The database's sharding strategy allows for transparent sharding of tables into tablets, which are replicated across multiple nodes. DocDB also supports leader leases to ensure that reads are served by the latest leader, and it uses Hybrid Logical Clocks (HLC) to track causal relationships between operations. The database provides a document-based persistence model, where each row is persisted in a local document store using a customized version of RocksDB. YugabyteDB aims to bring the Google Spanner architecture to life in an open-source, cloud-native infrastructure while retaining high-performance characteristics and reusing PostgreSQL's query layer.

YugabyteDB 1.2 is now officially available for download, marking the latest milestone in the development of YugaByte's distributed SQL API, YSQL, which offers PostgreSQL v11 compatibility and features such as low latency, internet scale, geographic data distribution, and extreme resilience to failures. The new release includes significant improvements, including a streamlined Kubernetes experience, Jepsen testing passed, and performance enhancements, with YugabyteDB delivering an average of 3.5x higher throughput and 3x lower latency compared to CockroachDB in benchmark tests. Additionally, the company has announced new customer wins, including Plume and Ink Aviation, which are using YugabyteDB for internet-scale microservices, IoT, and smart devices applications. The team is also hiring for various positions, including developer advocate, solutions engineer, software engineer, and more.

YugabyteDB is designed for high availability and absolute consistency, offering synchronous replication of writes to at least two nodes in the cluster, resulting in low latency strongly-consistent reads and automatic failover. In contrast, PostgreSQL uses asynchronous replication with a master-slave configuration, which can lead to higher latency and unavailability, while MongoDB's asynchronous replication approach also results in high latency for strongly consistent reads. YugabyteDB supports ACID transactions, including single row and multi-shard transactions, whereas PostgreSQL and MongoDB do not support distributed transactions on multiple shards. Overall, YugabyteDB is a high-performance distributed SQL database that combines the best of PostgreSQL and MongoDB into a single database.