January 2023 Summaries

The use of delete markers, also known as tombstones, in storage systems based on LSM1/MVCC can lead to scan performance degradation when the number of tombstones increases. This is because the query executor has to seek through the tombstones to find the earliest item, which can be time-consuming. To overcome this issue, a simple trick is to help the executor figure out an approximate point to start its search by using the last seen timestamp (`<last_ts>`) and modifying the query slightly. This approach reduces the number of seeks and significantly improves scan performance. Additionally, regular compaction can remove tombstones and prevent issues. However, in some cases, it may not be feasible to modify the application or handle scattered delete tombstones.

Yugabyte University has released a new self-paced course, YugabyteDB YCQL Development, designed to help developers learn about distributed SQL database development using Yugabyte Cloud Query Language (YCQL). The three-hour course covers the fundamentals of YCQL data definition language and data manipulation language, as well as exploring the internals of YugabyteDB. It aims to demystify key concepts like partition keys, clustering keys, and tablets, and provides a practical methodology for modeling semi-relational data through query-driven data modeling. The course includes hands-on lab exercises using Gitpod, VS Code Browser, and Jupyter notebooks, offering a frictionless learning experience with no-cost access to the necessary tools and services.

Trust, relationships, and outcomes are the core values of Yugabyte that make it an attractive company to work for. The author was drawn to the company's remote-first mindset and people-first approach, which prioritizes building strong relationships among team members. This emphasis on trust, communication, and teamwork led to a sense of belonging and fulfillment in their new role. By focusing on these core values, Yugabyte fosters an environment where employees can collaborate effectively, celebrate successes, and support each other's growth. The company's commitment to its core values has created a strong foundation for building trust, relationships, and outcomes, making it an excellent choice for those seeking a meaningful and fulfilling career.

YugabyteDB Voyager is a database migration tool designed to simplify and automate data migration from legacy and single-cloud databases to YugabyteDB, a distributed SQL database. The tool provides an end-to-end migration lifecycle, including planning, assessing, migrating, verifying, and application cutover and optimization. It supports offline and live migrations, as well as failback capabilities in case of failures. Voyager is free, 100% open source, and supports moving data to any version of YugabyteDB. The tool offers a unified CLI experience for all source databases, including Oracle and PostgreSQL, and can handle large-scale database migrations with minimal downtime. It has been successfully tested by beta customers, who have reported positive results in simplifying their database modernization initiatives.

Knex.js is a popular query builder for Node.js that supports multiple SQL dialects, including PostgreSQL. It provides an intuitive interface for building database queries without requiring the use of classes or abstracting away the underlying SQL syntax. Knex works seamlessly with YugabyteDB, a distributed SQL database built on PostgreSQL, making it an ideal choice for developers looking to scale their data layer. The library offers numerous benefits, including improved developer productivity, reduced errors, and enhanced application performance. Its simplicity and flexibility make it a great alternative to Object-Relational Mappers (ORMs) like Sequelize, TypeORM, and Prisma.

Turning a Data Modernization Strategy into a Data Monetization Program is about transforming data into a valuable asset for businesses. Fiserv, a company with extensive experience in application development and digital transformation, has a data-first approach and plans to monetize its data through various means such as offering data products and analytics services to merchants and consumers. The company's applications development team operates in an Agile scrum model, prioritizing developer productivity, automation, and self-service. To achieve this, Fiserv is exploring the use of modern platforms like YugabyteDB, container management platforms, and cloud-native technologies. The goal is to create a platform-as-a-service that exposes services for app developers, eliminating the need for them to build their own database expertise from scratch. This will enable faster development, better data consistency, high availability, and resilience. Fiserv also recognizes the importance of cultural changes to support this model, including embracing failure, transparency, and stake-setting. By doing so, they aim to create a "Tip of the Spear" team that is agile, productive, and able to deliver value quickly.

The author of this text is a member of the Yugabyte Quality Assurance (QA) team, responsible for testing the company's cloud-native database for business-critical enterprise applications. The QA team uses a wide range of testing frameworks and tests to ensure the database's correctness and reliability. They employ various testing approaches, including unit tests, cross-functional tests, stress tests, and randomized testing, to catch bugs before they reach customers. The team also utilizes open-source tools like libFuzzer, SQLancer, and SQLsmith to improve their testing efficiency. The goal is to provide a robust and reliable database system that can handle the demands of business-critical applications while providing continuous availability and horizontal scalability.

Testing Spring Boot applications with YugabyteDB using Testcontainers allows developers to write integration tests that mimic the production environment, providing a more realistic testing scenario. This approach uses Testcontainers, a Java library that provides lightweight, throwaway instances of common services running in a Docker container, to create a real database for testing purposes. By defining the library dependency and initializing the container programmatically or via the configuration file, developers can integrate Testcontainers into their Spring Boot applications and write integration tests that focus on the service layer and repository components. The process involves setting up prerequisites such as JDK 17 or above and Docker Engine, and then using annotations like `@SpringBootTest`, `@AutoConfigureTestDatabase`, `@Testcontainers`, and `@ActiveProfiles` to configure the application to use the real database through Testcontainers.

**YugabyteDB's Node.js Smart Driver offers a convenient, load-balancing solution for distributed applications. By leveraging the PostgreSQL query layer, it allows developers to distribute queries across multiple database nodes with minimal additional configuration. The smart driver handles node failures and rebalances traffic seamlessly, eliminating application downtime and reducing maintenance headaches. It also supports topology-aware load balancing, allowing requests to be routed within specific geographic regions or data centers. This eliminates the need for external load balancers and allows developers to focus on core application code, improving latency and productivity.

The text discusses the use of Monitoring Change Data Capture (CDC) using YugabyteDB's Metrics Dashboard to monitor the health, performance, and progress of CDC operations. The metrics dashboard provides an intuitive way to visualize the behavior of these operations, including important health metrics such as Change Event Count, Sent Lag, and Stream Expiry Time. The text also explains how to set up the metrics dashboards using Docker and Docker Compose, and how to use them to monitor CDC operations in various scenarios, including when a connector stops or is restarted. The dashboards provide valuable information for monitoring CDC operations, performance testing, and benchmarking.

Companies are migrating their mission-critical applications from legacy architectures to modern microservices-based architectures running on public, private, or hybrid cloud infrastructure. However, moving the stateful data layer at the same speed as the stateless application layer is a major challenge. Legacy SQL-based relational databases are unsuitable for dynamic cloud infrastructure, while NoSQL and NewSQL databases present significant operational challenges when run at scale in production. Distributed SQL databases, like YugabyteDB, offer a solution to these challenges by providing strong consistency, horizontal scalability, geographic distribution, SQL API, automatic data replication, distributed query execution, and support for distributed ACID transactions. These characteristics enable companies to deliver high-value applications and services more quickly, while lowering IT costs and reducing risk through compliance.

A leading US-based omnichannel retailer is modernizing its digital platforms to better support its customer-centric culture, aiming to find new ways to engage with customers while keeping its IT architecture simple, resilient, and scalable. The retailer has undergone several digital transformations, including moving from an old ASP.net application to a headless architecture and adopting cloud-based Spring Boot microservices deployed with Kubernetes. To optimize its e-commerce platform, the retailer chose YugabyteDB as the database for key workloads such as promotions engine, catalog, customer accounts, registries, and card and checkout. By modernizing its core e-commerce platform, the retailer expects faster website and mobile app performance, more resilient services, and lower operating costs. The database plays a crucial role in supporting the retailer's microservices-based architecture, allowing for easy scaling, deployment of changes, and targeting improved performance of individual services. With YugabyteDB, the retailer achieved low latency and high availability, meeting its performance-at-scale requirements and enabling faster deployments and independent team work.