October 2018 Summaries

YugabyteDB's YSQL is designed to support PostgreSQL-compatible distributed SQL workloads while leveraging the existing open-source Postgres codebase for a high degree of support and accelerated time-to-market. YugabyteDB achieves this by utilizing its common distributed storage engine, which powers both SQL and NoSQL applications, allowing it to scale out linearly on demand. The system uses DocDB as a persistent "key to object/document" store on each node, supporting efficient fine-grained attribute updates and lookups while handling many terabytes of data per node. YugabyteDB's YQL layer implements the server-side of multiple protocols/APIs that support various use cases, including PostgreSQL wire-protocol for production usage. The system supports distributed ACID transactions, enabling features such as strongly consistent secondary indexes and multi-table/row ACID operations in the YSQL context.

YugabyteDB has introduced YSQL, its PostgreSQL-compatible distributed SQL API, as part of the 1.1 release. This development aims to simplify operational database infrastructure by bringing together the best aspects of SQL and NoSQL into a single transactional, high-performance database. The new architecture offers several benefits, including scale-out on-demand, resilient self-healing, high performance at massive scale, multi-zone and geographically replicated deployments, cloud-native and Kubernetes-ready capabilities. By building YSQL on top of the YugabyteDB core engine, developers can now leverage PostgreSQL functionality while taking advantage of the scalability and performance advantages offered by distributed SQL. This move is part of YugabyteDB's mission to simplify operational database infrastructure, making it easier for developers to build fast-growing online services that require agility and feature velocity.

Apache Cassandra is designed as an eventually consistent database that allows per-operation tradeoff between consistency and availability through tunable consistency levels. However, this approach does not support multi-shard ACID transactions, and its features such as quorum writes/reads, lightweight transactions, and secondary indexes are not suitable for achieving single-key ACID guarantees. Instead, these features can lead to poor performance and inconsistent data. Cassandra's eventual consistency model is based on a BASE paradigm that prioritizes availability over strong consistency, which means it sacrifices some level of consistency to achieve higher availability. This approach is fundamentally different from traditional database systems like Oracle or IBM DB2 that follow the ACID paradigm, which guarantees atomicity, consistency, isolation, and durability. For use cases requiring strong consistency, low latency, and high density, newer databases like YugabyteDB offer a more transactional approach that solves the problems associated with Cassandra's eventual consistency model.

Google Spanner and Calvin are two approaches to achieving global consistency at scale in distributed databases. While both systems aim to provide strong consistency, they differ significantly in their design and implementation. Google Spanner uses a partitioned consensus approach with TrueTime-based clock skew tracking, which allows it to guarantee correctness of multi-shard/distributed transactions under bounded clock skew scenarios. On the other hand, Calvin uses a global consensus approach that avoids clock skew tracking by preprocessing transactions, resulting in higher write throughput for certain workloads. However, this comes at the cost of lower availability and increased latency for writes. Additionally, Calvin's restrictive transaction model makes it unsuitable for building SQL-compatible databases. Spanner, on the other hand, supports full SQL complexity and has better performance for read-only transactions and general-purpose high-volume concurrent transactions. Overall, the choice between Spanner and Calvin depends on the specific use case and requirements of the application, with Spanner being a better fit for applications requiring strict serializability isolation level and Calvin being more suitable for distributed-transaction-heavy workloads with low conflict access to same data in concurrent transactions.

YugabyteDB 1.1 introduces a new feature that simplifies multi-cloud, hybrid cloud, and multi-region database deployments by providing public IPs to enable easy communication between servers across different regions and clouds. This feature allows users to specify private IP addresses for internal server-to-server communication within a region or cloud, while using public IP addresses for inter-regional or cross-cloud communication. The new flags `rpc_bind_addresses` and `server_broadcast_addresses` are used to configure this functionality, which also includes optional settings such as the `use_private_ip` policy to determine when to use private IP addresses. This feature is designed to address common challenges in multi-cloud and hybrid cloud deployments, where cost is a significant factor in choosing between public and private IP addresses for communication. The new feature provides a more efficient way to deploy and manage databases across different regions and clouds, making it easier to achieve scalability, flexibility, and cost-effectiveness.