March 2020 Summaries

Hasura, an open-source GraphQL engine, connects to databases and microservices and generates a production-ready GraphQL backend. YugabyteDB, a distributed SQL database, provides high performance and resilience when combined with Hasura. To get started with Hasura and YugabyteDB on Google Kubernetes Engine (GKE), users need to set up a Kubernetes cluster, install YugabyteDB using Helm, create a sample database, and configure Hasura to use YugabyteDB. Once set up, users can perform basic CRUD operations such as querying data, adding new entries, updating existing entries, and deleting entries using GraphQL. With Hasura and YugabyteDB, developers can easily build applications and microservices against the database, taking advantage of its benefits including high performance, resilience, and scalability.

In the realm of databases, relational database management systems (RDBMS) like Oracle, PostgreSQL, and MySQL have been prevalent for decades, leveraging SQL as their lingua franca. However, NoSQL databases such as MongoDB and Apache Cassandra emerged in the mid-2000s, initially positioned as alternatives to RDBMS but later evolving to coexist alongside them. The distinction between SQL and NoSQL has blurred, with many NoSQL databases now incorporating ACID transactions. NewSQL databases were introduced to address scalability challenges, with two main flavors: automated data sharding on top of monolithic databases or new distributed storage engines designed from the ground up. With the rise of cloud-native applications and global distribution, geo-distributed SQL databases have emerged as a solution, offering horizontal scalability, resilience, and support for fully-distributed ACID-compliant transactions. Cloud-native databases like YugabyteDB are now leading the pack, enabling multi-API, multi-model application development on top of a single transactional storage engine.

The Yugabyte team has been working from home due to social distancing efforts, but they have transitioned to online meetings with customers and partners. The YugabyteDB Helm Chart is compatible with both Helm 2 and Helm 3, but the syntax and setup differ between the two versions. The recommended way to deploy YugabyteDB in a Red Hat OpenShift environment is using the Helm chart, requiring admin deployment privileges and sufficient CPU/memory resources on Kubernetes nodes. Minikube users can access YSQL services by running `minikube tunnel` in a separate command, while launching the AWS CloudFormation template with an existing VPC requires replacing the 'VpcId' reference with the desired VPC ID. New documentation, blogs, tutorials, and videos have been published, including guides on data modeling, deploying YugabyteDB on Google Kubernetes Engine clusters, and getting started with pgbench and distributed SQL on GKE. The company is hiring for various positions, including development representatives, site reliability engineers, community success engineers, and software engineers.

YugabyteDB has two JSON data types, `json` and `jsonb`, which can be used to store documents in a column in a YSQL table. The `jsonb` data type stores a parsed representation of the document hierarchy of subvalues in an internal format, making it more efficient for operations such as indexing and querying. YugabyteDB supports various JSON operators and functions, including content-based queries, shreds, shred operations, and index support. The database also supports constraints on JSON expressions, allowing developers to enforce business rules on the data stored in JSON documents. An example is provided of a function `chk_book_doc_is_conformant` that checks if a document conforms to certain expected structures, demonstrating how to create a constraint on a PL/pgSQL function that encapsulates rich JSON functionality.

YugabyteDB is an easy-to-use distributed SQL database that can be deployed on various infrastructure platforms, including public cloud providers like AWS. The Yugabyte team has been working remotely due to the COVID-19 pandemic and transitioning to online meetings with customers and partners. To ensure service availability, YugabyteDB's architecture is designed for resilience, scalability, and geo-distribution in dynamic environments such as VMs and containers. AWS CloudFormation templates can be used to automate YugabyteDB deployments, providing a consistent manner of deploying public cloud workloads. The templates store in plain-text YAML format and include settings such as replication factor, key name, VPC, and security group configuration. To deploy YugabyteDB using CloudFormation, users need an AWS account with permissions for creating EC2 instances and utilizing CloudFormation, and can follow the deployment steps outlined in the example.

The text discusses the use of `pgbench`, a simple program for running benchmark tests on PostgreSQL, to compare its performance with that of YugabyteDB, an open-source, high-performance distributed SQL database. The authors have forked `pgbench` and renamed it to `ysql_bench` to run against YugabyteDB, which is PostgreSQL wire compatible. The tutorial guides the reader through setting up a Kubernetes cluster on Google Cloud Platform (GCP) and deploying YugabyteDB using Helm charts. It then walks through initializing and running `ysql_bench` against the YugabyteDB cluster, providing an overview of its usage and configuration options. The authors highlight that while `pgbench` is designed for single-node PostgreSQL deployments, it can still be used to benchmark distributed SQL databases like YugabyteDB, offering valuable insights into cluster sizing, deployment architecture, and resource allocation.

YugabyteDB has released version 2.1.2, which includes over 30 new enhancements and fixes, covering various aspects of the database, including core, change data capture, YSQL API, and the Yugabyte platform. The team has been working remotely due to the COVID-19 pandemic and is continuing with online meetings and social distancing measures. The database is built on a scalable and fault-tolerant design inspired by Google Spanner and provides a PostgreSQL wire-compatible SQL API. The release notes highlight various bug fixes, new features, and changes in the database, including improvements to transactions, CDC streams, and YSQL API. Additionally, YugabyteDB offers resources for getting started with the database, including documentation, tutorials, blogs, videos, and job openings.

Distributed SQL is a key feature of YugabyteDB, enabling the use of SQL across multiple nodes in a distributed system. YugabyteDB provides several options for optimizing performance in integration testing, including using ramdisk directories, reducing shards per tserver, and setting replication factor to 1. The YSQL equivalent of SQL Server's ISNULL() function is COALESCE(), and duplicate rows can be found using GROUP BY and HAVING clauses, or by using DISTINCT ON to remove them. YugabyteDB also provides new documentation on read replica clusters, JSON data types, and a quickstart page for getting started with the database. Additionally, there are new blogs, tutorials, and videos available on various topics related to YugabyteDB.

Prometheus is a popular open-source time-series metrics monitoring solution that has matured since its inception in 2012 and is widely used for monitoring Kubernetes cluster metrics and container-based applications. It can be easily integrated with Grafana for visualization, providing a powerful combination for dashboarding application performance. YugabyteDB, an open-source distributed SQL database, exposes Prometheus endpoints for easy metrics collection from its inception. The Yugabyte Platform and Cloud offerings have extensive monitoring features built on Prometheus. To deploy Prometheus and Grafana in a Kubernetes cluster, one can use either the Kubernetes prometheus-operator or create a straightforward YAML manifest, specifying the container image, ports, protocol, volume mounts, and configuration files. The Grafana deployment is similarly simple, consisting of a single container specified in a YAML manifest with volume mounts and environment variables. Once deployed, Prometheus and Grafana can be used to monitor application performance, identify bottlenecks, and generate synthetic workloads for testing purposes. The YugabyteDB-powered Hipster Shop demo provides a real-world example of deploying monitoring components. Future work includes expanding the monitoring scope to include Kubernetes cluster metrics and other Prometheus-compatible services, as well as centralizing monitoring via CNCF's Cortex project.

The text discusses Falco Runtime Security and Cloud Native Distributed SQL on Google Kubernetes Engine (GKE). It highlights how Falco, an incubating CNCF project, provides cloud native runtime security for applications running in Kubernetes environments. The text also introduces YugabyteDB, a cloud native, distributed SQL database designed to run in Kubernetes environments, which can interoperate with Falco and other CNCF projects out of the box. The authors aim to show how to get up and running with YugabyteDB and Falco on GCP, implement basic security policies, and demonstrate the integration by creating a test user and logging into the database, resulting in an audit log entry from Falco.

YugabyteDB is a distributed SQL database that supports snapshot isolation and serializable isolation levels, allowing developers to configure the desired level of concurrency and consistency for their applications. The database also allows users to create read-only accounts with limited permissions, enabling secure access to data without compromising performance. YugabyteDB uses DocDB as its persistence layer and employs the Raft algorithm to ensure data replication across nodes in the cluster. While auto-incrementing IDs can introduce bottlenecks in distributed SQL databases, using UUID columns or serial columns with caching can provide more efficient alternatives. The database also supports various features such as colocated tables, longhorn storage, and integration with Kubernetes. Developers can explore YugabyteDB's capabilities by following tutorials, watching videos, or attending upcoming events.

This demo application showcases how distributed SQL can improve microservices architecture by providing a single, scalable database solution for polyglot persistence. The application is built on top of Kubernetes and uses YugabyteDB as the distributed SQL database, replacing the original polyglot persistence implementation with independent database instances. By leveraging distributed SQL, developers can simplify their development and operations workflows while achieving better performance, scalability, and availability. The demo application demonstrates how to deploy and integrate the application with Istio for service discovery and communication, and provides a starting point for building cloud-native applications that benefit from distributed SQL.

In YugabyteDB, pushdowns are used to improve performance by moving query processing as close to the data as possible. This reduces network traffic and improves query latency. Pushdowns can be applied to various queries, including single row operations, batch operations, filtering using index predicates, expressions, and index-organized tables. The goal is to reduce remote RPC calls and optimize data transfer over the network. By pushing down these optimizations, YugabyteDB achieves significant performance gains, ranging from 3x to 15x improvements in throughput and latency. Ongoing work includes further enhancements, such as pushing down entire execution subtrees, optimizing GROUP BY queries, and improving parallel queries.