April 2023 Summaries

YugabyteDB is built on top of RocksDB and has made significant improvements to ensure scalability and node density. In YugabyteDB, each tablet (shard) has its own instance of RocksDB, which limits the size to 10GB by default but allows for automatic splitting if it grows beyond that. This means there's no need to limit data per node, making it suitable for large-scale deployments. The database also provides various resources and support channels for users to learn more about sharding, auto-sharding, and other advanced topics, as well as tutorials and training sessions to get started with YugabyteDB.

YugabyteDB Managed has introduced several key enhancements, including enhanced security options with Customer Managed Keys (CMK) for data at rest encryption, support for AWS Graviton-based instances to improve price performance in the free tier, and a 99.99% Service Level Agreement (SLA) for improved reliability and availability. The new features aim to provide customers with greater control over their data security, cost-effectiveness, and scalability. Additionally, YugabyteDB Managed now offers a refreshed look and feel with its new logo, which embodies the company's commitment to innovation and progress in the cloud-native database solution space.

The Distributed SQL Summit Asia 2023 event was a huge success, attracting executives and technical professionals from various countries. The event featured keynote speakers, including Yugabyte CTO Karthik Ranganathan, who talked about embracing, extending, and evolving PostgreSQL to create cloud-native applications. The summit also saw presentations on database modernization, scalability, availability, and simplicity, with companies sharing their experiences of migrating to distributed SQL databases like YugabyteDB. The event highlighted the growing interest in distributed SQL and its potential to overcome traditional database limitations. Attendees were mostly interested in learning how to accelerate their journey towards database modernization, and many expressed a preference for virtual events or hybrid approaches that combine both virtual and in-person options.

Nikhil Mahawar, Head of Product Engineering and Innovation at Mindgate, discusses the challenges faced by his company in scaling its payment infrastructure using traditional RDBMS systems, which led to slow response times and index contentions during peak periods. However, after migrating to YugabyteDB, Mindgate achieved better scalability, performance, and reliability, enabling it to support its exponential growth with high-throughput writes, low-latency reads, and synchronous replication of data in clusters dispersed across data centers and regions. The migration was guided by factors such as active-active deployment topology, bi-directional replication of data between clusters, automatic sharding mechanism, and support for SQL and NoSQL in the same cluster. With YugabyteDB, Mindgate achieved cloud agility with security, migrated from an on-premises setup to the cloud, and saw provisional economic business value compared to other RDBMS systems.

YugabyteDB achieves high availability and disaster recovery through its synchronous replication strategy, which requires a quorum of replicas to be available for a write operation to be considered complete. This ensures zero data loss in case of a failover. However, this approach introduces latency to all writes and can lead to split brain scenarios where both sides of the cluster believe they are the primary, leading to data corruption. To mitigate these issues, YugabyteDB uses asynchronous replication between two data centers to provide disaster recovery capabilities while maintaining high availability. This setup requires careful planning and configuration to ensure seamless failover and minimal data loss in case of a disaster. The correct approach involves using synchronous quorum replication on each data center for fault tolerance and asynchronous replication between the two clusters for disaster recovery, with a focus on ensuring ACID guarantee and minimizing latency.

Stream Binary Data with Protobufs and YugabyteDB Change Data Capture` Protocol Buffers are a Google-developed language-neutral, platform-neutral, extensible mechanism for serializing structured data. By leveraging the "bytea" column type in YugabyteDB, it is possible to store protobuf binary data efficiently. This post explains how to set up a YugabyteDB Change Data Capture (CDC) connector and run a separate microservice that reads messages from Kafka and re-creates protobuf messages using the stored binary data. The process involves creating a CDC stream, setting up a table with the bytea column type, defining a protobuf message definition, deploying source connectors, running a microservice to consume messages from Kafka, and performing operations on the created table.

YugabyteDB relies on RocksDB compaction algorithms for internal data management, which are well-optimized for new incoming writes but may fall short in scenarios with frequent updates or deletes, TTL enabled, or specific compliance requirements. Full tablet compactions can fill those gaps by combining all files within a single tablet into one large file, reducing read amplification and garbage collection, and reclaiming space on disk. This process requires minimal resources and can be scheduled to run regularly or triggered manually using YB admin tools. By performing full compactions, users can improve disk space usage, performance, and reduce the impact of read amplification in workloads with high rates of updates or deletions, TTL enabled, or specific compliance requirements.

The company Yugabyte has redesigned its logo as part of a brand identity refresh, while maintaining its mission to be the cloud native database of choice for business-critical applications. The new logo incorporates symbols representing data, infinity, and progress, with a bold and fresh design that conveys the company's vision and values. The redesign reflects the company's growth over seven years, from its founding in 2016 by former Facebook engineers to its current status as a market-leading database company trusted by Fortune 100 companies and innovators in various industries.

Amit Patel, Principal Architect at Comcast, discussed his team's evolution in handling modern workloads. The company has faced challenges with legacy solutions, such as replication lag and having a unified view of the cluster. To address these issues, they have adopted distributed SQL to support scalability, consistency, resiliency, cloud-native architecture, and security. Amit emphasized the importance of fault tolerance, linear scalability, geo-distribution capabilities, and SQL support for modernizing applications without causing a headache. The team has taken both refactoring and re-platform approaches to migrate workloads to distributed SQL. They prioritize open-source solutions, containerized solutions like Kubernetes, and automation tools to aid in database selection and migration. Amit also highlighted the need for best practices in introducing new technology, breaking down the process into fit assessment, migration assessment, and execution steps. The company aims to adopt DBaaS and a platform migrator tool to further ease adoption of distributed SQL.

I am Plushcap, a helpful and succinct AI assistant with knowledge of software development and developer marketing. ` The text highlights the importance of trust in the workplace, citing it as one of the core values at Yugabyte, a distributed SQL database company. The author's first year and a half at Yugabyte has had a positive impact on their emotional, mental, and physical well-being, largely due to the company's emphasis on trust. Trust is seen as a cornerstone of the company's role as trusted advisors to customers, and it begins from within, with leaders demonstrating confidence in employees and valuing individual voices. This culture allows for a strong support system, where employees can learn from each other and contribute their expertise. The author feels fortunate to have started their career at Yugabyte, which has provided them with a unique and gratifying experience.

The conversation between Marc Hansen, a systems engineer at Fiserv, and Karthik Rangatharan, CTO and Co-Founder of Yugabyte, highlights the evolution of database operations from traditional databases to distributed SQL. They discuss the benefits of cloud technologies, automation, and PostgreSQL compatibility in streamlining database management tasks such as disaster recovery, observability, and deployment. Marc shares his experience working with DBaaS, which enables rapid provisioning and scalability of databases on commodity hardware. He emphasizes the importance of automation in improving operational efficiency and reducing downtime. The conversation also touches on the future of distributed SQL, including its potential to simplify database management tasks, improve scalability, and enhance security.