March 2021 Summaries

Creating Standard Patterns to Load Data into YugabyteDB with Apache Airflow` presents a talk by Viraj Parekh and Pete DeJoy from Astronomer, where they showcase how data platform engineers can standardize business-specific actions using Apache Airflow to replicate Postgres data to a distributed YugabyteDB instance. They demonstrate the use of Airflow's Providers Framework, which enables users to connect, transfer, manipulate, and analyze their data with pre-built components for various services like Google Cloud, Amazon Web Services, and Microsoft Azure. The talk highlights the flexibility of Airflow in handling legacy on-prem systems and cloud-native applications, making it an essential tool for data teams undergoing digital transformations. By providing a standard template for users to perform business-specific actions, Airflow enables data platform engineers to maintain, reuse, and extend their workflows efficiently.

YugabyteDB and Yugabyte Platform have received updates in their documentation, including the addition of new content to the Explore section, which provides an intuitive design to help users get started with YugabyteDB and try out its latest features. The documentation now covers topics such as YSQL vs PostgreSQL, fault tolerance, horizontal scalability, transactions, JSON support, multi-region deployments, query optimization, follower reads, colocated tables, change data capture (CDC), observability, and more. Additionally, the documentation for Yugabyte Platform has been updated to provide a streamlined operational experience, including features such as automated deployment, day-2 operations, and services needed to realize business outcomes with the database. The documentation is designed to help users explore both platforms and contribute to its development through GitHub issues and contributions.

A large Fortune 500 financial services company has chosen YugabyteDB as its distributed SQL database to support its digital transformation initiatives, including migrating away from mainframes and adopting a microservices-oriented cloud native architecture. The company required a highly available, scalable, and secure database that could handle trillions of dollars in assets and provide strong data consistency and ACID compliance for distributed transactions. YugabyteDB was selected due to its ability to be deployed across data centers and the public cloud, providing a fresh look at their underlying data tier. After deploying YugabyteDB, the company has seen positive results, including zero downtime, 10x performance gain, and improved productivity, while reducing costs by transitioning workloads to commodity hardware.

DBeaver Enterprise edition now supports first-class integration with YugabyteDB Cloud Query Language (YCQL), allowing native querying of YugabyteDB tables and JSONB data type. The support is available starting from DBeaver EE 21.0.0, and it enables users to work with the YugabyteDB YCQL Java driver. To get started, users need access to a YugabyteDB cluster, which can be deployed on a public cloud or using a DBaaS offering like Yugabyte Platform. The integration allows users to launch DBeaver EE, configure a connection to YugabyteDB YCQL, and explore the database navigator to work with the sample dataset and run queries on the JSONB data type.

I worked at Nutanix for five years, where I used and built nearly a dozen different databases. This experience made me realize how challenging it is to build a database that meets all the requirements. Yugabyte caught my attention with its ability to address reliability, consistency, query layer, transactions, and scalability without performance costs. As someone who wanted to contribute to open source, Karthik suggested I join Yugabyte's team, which was supported by Nutanix Legal. I started contributing to YugabyteDB by exposing Prometheus metrics, then worked on Row Level Geo-Partitioning project, enabling PostgreSQL's Table Partitioning feature in YugabyteDB. The codebase was neat and easy to navigate, with a comprehensive set of unit tests for command-line tools. Working with the team was enjoyable, with talented engineers who were patient and appreciated my contributions. After showcasing my work on Row Level Geo-Partitioning, I received an offer from Yugabyte and started working there, where I'm now working on completing this project and contributing to the company's success.

Yugabyte has secured $48M in funding, bringing its total funding to $103M, to accelerate the launch of its fully managed cloud offering and expand into new markets such as EMEA and APAC. The company's open source distributed SQL database, YugabyteDB, is seeing rapid adoption by enterprises and startups alike, driven by its permissive Apache 2.0 open source licensing, comprehensive set of PostgreSQL-compatible features, and support for multi-cloud deployments out-of-the-box. With the new funding, Yugabyte plans to grow its R&D, sales, and customer support functions, and welcomes new customers such as Hudson River Trading. The company is also expanding its global presence through hiring more staff in India, Russia, Canada, EMEA, and APAC, with a focus on building a cloud-native distributed SQL database that combines the horizontal scalability of NoSQL systems with the ACID guarantees of a SQL system.

The text discusses the use of PostgreSQL's recursive common table expression (CTE) to traverse hierarchical data and solve problems like the "Bacon Numbers" problem. It explains how YugabyteDB, a distributed SQL database, re-uses the PostgreSQL source code to provide SQL and stored procedure functionality. The author uses examples, including an employee hierarchy and the Bacon Numbers problem, to illustrate the use of recursive CTEs in solving these types of problems. The text also covers the syntax and semantics of the WITH clause and common table expressions (CTEs) in YugabyteDB's YSQL documentation. It provides code snippets and explanations of how to implement recursive CTEs using YugabyteDB's YSQL, including examples of breadth-first and depth-first traversals of a hierarchy. The author concludes that the power of PostgreSQL's recursive CTE is awe-inspiring and invites readers to learn more in Part Two.

The key points of the text are that the author, who is familiar with PostgreSQL and YugabyteDB, presents a solution to the Bacon Numbers problem using recursive Common Table Expressions (CTEs) in both databases. The Bacon Numbers problem involves finding the shortest chain of connections between two actors in a movie database, where an actor's Bacon Number is the length of the shortest chain connecting them to Kevin Bacon. The author explains how to use recursive CTEs to solve this problem in both PostgreSQL and YugabyteDB, highlighting similarities and differences between the approaches used for hierarchical and cyclic graphs. They also discuss the importance of early pruning in solving large-scale problems like this one.