June 2021 Summaries

YugabyteDB is a distributed, fault-tolerant, and highly available database with extremely low latencies for reads and writes. Its cluster balancer automatically balances the load across multiple nodes, sharding data into tablets and replicas to ensure uniform distribution and increased throughput. The balancer can be controlled via knobs, and features include per-table balancing, global balancing, replication factor management, user-defined data placement policies, leader draining, rebalancing on node failures, and support for throttles. However, there is ongoing work to improve the load balancing algorithm to consider actual tablet loads and ensure proper balance across nodes with multiple disk drives.

Hasura Cloud and YugabyteDB Managed are integrated to provide a seamless experience for building real-time polling applications. Hasura's GraphQL Engine connects to the database, auto-generating a production-ready API backend, while YugabyteDB offers horizontal scalability, fault tolerance, and global data distribution. The two services can be deployed on any public or private cloud, with YugabyteDB being the only distributed SQL database to provide integration and support for GraphQL applications. A real-time polling app built using React is powered by Hasura GraphQL Engine backed by a YugabyteDB Managed database cluster, showcasing the simplicity of their technology integration. The process involves creating free accounts on both services, downloading the sample application code, connecting to the database, applying migrations, configuring the app, and running it, resulting in a fully functional real-time polling application with automatic data updates using GraphQL Subscriptions.

YugabyteDB and Hasura have partnered to integrate their platforms, making it easier for developers to build complex, global applications that meet user expectations for availability, speed, and experience. With this integration, developers can deploy a highly available, massively scalable, geo-distributed database from Hasura Cloud to Yugabyte Managed with just one click. This allows for instant GraphQL APIs, easy deployment of GraphQL applications, and optimized data transfer. The partnership aims to simplify the process of building innovative apps that deliver on user demands, leveraging the strengths of both platforms to create a fast and easy platform for complex global applications.

YugabyteDB is a CP database designed to support two interfaces: YSQL for SQL workloads and YCQL for Cassandra-like workloads, with YCQL being the primary focus of this article. The database supports consistent reads by default and offers alternative types of reads, including follower reads that distribute read workload across all replicas hosting data and observer reads that use a separate cluster for read replicas, allowing for low latency access to far-off users and tuning analytics workloads. With YugabyteDB, applications can set the consistency level to ONE to get either follower or observer reads, ensuring timeline-consistency and controlling staleness with the `max_stale_read_bound_time_ms` GFlag. The database also provides fault tolerance through its RAFT group, which can tolerate k failures by having 2k+1 replicas, and follows different semantics when switching from observer reads to follower reads.

I joined Yugabyte as a Software Engineer in the Yugabyte Cloud team on June 1, 2021. Throughout my undergraduate studies, I completed multiple software engineering internships and explored diverse domains, which led me to discover my interests in databases, distributed systems, and cloud. I was drawn to Yugabyte due to its innovative approach to developing high-performance distributed SQL databases, as evident from a talk by Karthik Ranganathan, the CTO and co-founder of Yugabyte. I joined YugabyteDB Managed, which offers a Database-as-a-Service platform, allowing me to work on cloud-related features and contribute to the product's growth. The team is comprised of exceptionally talented engineers with years of experience working on databases at top companies like Facebook and Oracle, providing an enriching environment for learning and growth. I had a smooth and enjoyable interview process, where I was assessed not only as a problem solver but also as someone who can learn, grow, and explore diverse areas. Yugabyte's commitment to open source and community involvement is commendable, making it an ideal fit for me. Overall, joining Yugabyte has been an exceptionally exciting experience, offering a tremendous learning opportunity with a supportive and highly motivated team.

Migrating a MySQL database to YugabyteDB can be done using the pgloader tool, which provides a PostgreSQL-compatible distributed SQL database. To start the migration process, prerequisites such as accessing the MySQL database and having ysqlsh command line connectivity to a running YugabyteDB cluster are needed. A separate machine is recommended for installing and running pgloader to avoid stealing resources from the target or source servers. The pgloader instance needs to be able to connect to both the MySQL source database and the YugabyteDB target cluster, with necessary permissions granted. Once the setup is complete, a command file can be used to simplify the docker command, and the pgloader can be run to migrate the schema and data from MySQL to YugabyteDB. The migration process can also be done for just the schema, and validation steps include checking the platform UI for the presence of tables and active queries.

YugabyteDB provides a connection pooling mechanism that allows users to reuse the PostgreSQL query layer, making it easier to integrate with existing third-party integrations. Users can connect to their Yugabyte Cloud Database using the YSQL API and various IDEs and drivers, such as DBeaver and DataGrip. The database supports SERIALIZABLE and SNAPSHOT transaction isolation levels, which are mapped to SQL isolation levels REPEATABLE READ and SERIALIZABLE, respectively. Optimistic locking is currently supported, while pessimistic locking is being worked on. Adding a column to a large table in YSQL can be done using an ALTER TABLE command, but there may be limitations for non-nullable columns with default values. YugabyteDB has new documentation, blogs, tutorials, and videos available, including fine-grained control for high availability, people and potential stories, and upcoming events such as the Distributed SQL Summit Asia 2021.

Gavin, the author, joined Yugabyte as a product marketer in May 2021 after being unexpectedly unemployed for the first time in his life. He previously worked at RudderStack, New Relic, and AT&T, and has a background in IT system administration and development. Despite having received job offers from other companies, including Twilio-Segment, Gavin chose to join Yugabyte due to its promising product offerings, huge market opportunity, stellar executive team, friendly and collegial interview process, and alignment with his personal values of open source software and doing work that allows him to build technical content.

The YugabyteDB Vagrant Box allows users to easily test and run a YugabyteDB cluster without installing and configuring it manually, using Vagrant on Windows, Linux or Mac OS X. The box is non-official for demo purposes only, but provides a quick way to start and scale the cluster as needed. To use the box, users must have Virtualbox and Vagrant installed, with a reasonable internet connection and enough disk space. The YugabyteDB cluster can be configured using environment variables, including replication factor, number of machines, memory, and CPUs. Users can manipulate virtual machines using various Vagrant commands, such as `vagrant up`, `vagrant halt`, and `vagrant destroy`.