August 2022 Summaries

The blog post by Jeffrey White explores the feasibility of running CockroachDB in unconventional environments, specifically on IBM S/390 and ARM64 CPU architectures. The author details the process of setting up a 12-node CockroachDB cluster across three different regions, each using a distinct CPU architecture, cloud provider, and Linux operating system. The goal is to test the interoperability of CockroachDB in these diverse settings rather than optimizing for performance. The setup involves compiling CockroachDB from source to obtain the necessary binaries and using a virtual software-defined network for connectivity. Benchmark tests are conducted to ensure the system functions adequately, demonstrating that despite the heterogeneity, the CockroachDB cluster can operate as a single database. The blog highlights the adaptability of CockroachDB in different environments, suggesting its potential for simplifying deployments.

CockroachDB's engineering team will present their latest research paper, "A Demonstration of Multi-Region CockroachDB," at the VLDB 2022 conference, which will take place in Sydney, Australia, and online from September 5-9. The paper highlights the capabilities of CockroachDB in supporting multi-region database deployments with a high-level declarative syntax, simplifying the configuration of databases to meet modern applications' consistency, performance, and availability requirements. The demonstration will use a fictional ride-sharing company, movr, to showcase how CockroachDB's SQL syntax allows for easy conversion from a single-region to a multi-region database while maintaining strong consistency, low latency, and high availability. Presentations will be made by Arul Ajmani online and by Rebecca Taft and Oliver Tan in person at the conference.

In the context of test data management (TDM) and the increasing importance of data masking, the integration of CockroachDB and Delphix is highlighted as a solution to overcome traditional challenges associated with managing and securing data in development environments. The text discusses the historical issues of TDM, such as slow provisioning, inadequate data quality, and rising costs, and emphasizes the need for efficient tools that secure data without compromising agility. Delphix is presented as a key tool for integrating with CockroachDB, offering continuous data and compliance capabilities to automate and secure DevOps environments. The integration aids in data masking, application development, and legacy application migration, ensuring that sensitive data is protected while maintaining high-quality test data. This approach addresses the challenges of distributed databases by providing a seamless and scalable TDM solution, ultimately enabling faster software delivery and enhanced data security in compliance with regulatory requirements.

In her blog post, Marylia Gutierrez discusses the importance of indexes in database schemas for enhancing workload performance by allowing efficient data retrieval without scanning entire tables. She warns against excessive index creation due to potential negative impacts on write performance, as data must be written for all indexes on a table. Gutierrez highlights CockroachDB's features that aid in analyzing index usage, thereby offering recommendations on creating, dropping, or updating indexes. She explains how to use the Statements Page to avoid full scans, which are costly for large tables, and suggests employing the EXPLAIN command for index recommendations. Additionally, she mentions using Index Statistics to identify underutilized indexes that may hinder performance, advising regular monitoring and adjustments as data and access patterns evolve. Gutierrez also notes upcoming updates to CockroachDB that will further streamline index management and encourages users to leverage support resources for performance issues.

Edgar F. Codd, whose 99th birth anniversary was marked in August 2022, revolutionized data management through his development of the relational database model, a concept that disconnected data access methods from the data itself, making databases more accessible and user-friendly. Before Codd's intervention, databases were cumbersome, requiring specialized skills and understanding of data's physical storage. His 1970 paper, "A Relational Model of Data for Large Shared Data Banks," introduced a new framework that used simple rows and tables, which laid the groundwork for modern database systems and influenced developments such as SQL. Despite initial resistance from IBM, where he worked, Codd's ideas led to the creation of System R, an experimental relational database that demonstrated the practicality of his model. Although he never directly benefited financially from his innovations, Codd's work laid the foundation for an industry worth billions today and earned him the 1981 A. M. Turing Award, underscoring his significant impact on data processing and everyday technology use.

Netflix has developed a robust infrastructure to ensure seamless streaming experiences across a wide array of devices, driven by their commitment to enhancing customer satisfaction and fostering a culture of experimentation. The company's Device Management Platform, managed by the Partner Infrastructure Team, is pivotal in supporting this strategy by maintaining compatibility with numerous devices, such as smart TVs and streaming sticks from partners like Roku, Samsung, and LG. This platform employs a cloud-based automation framework called Netflix Test Studio to manage device testing, ensuring high streaming quality and user satisfaction. By leveraging technologies like MQTT, Kafka, and CockroachDB, Netflix achieves fault tolerance and horizontal scalability, crucial for managing device information and processing events. This infrastructure not only meets current demands but is also designed to scale with the growing number of devices and users, setting an industry standard for adaptive and future-ready application architectures.

The challenges of media streaming, particularly during high-demand events, are significant and reminiscent of the infamous 1968 "Heidi Bowl" incident, where technical limitations led to viewers missing the end of a live sports broadcast. Today, media streaming companies face similar issues when handling "hot content," or highly popular media that attracts massive simultaneous viewership, which can strain infrastructure and lead to service disruptions. The architecture of streaming services relies heavily on metadata storage to manage user access and deliver the correct content, which can become overwhelmed by surges in demand. To address these challenges, companies use caching systems to manage load but face potential issues with outdated or incorrect data being served. CockroachDB is highlighted as a solution, offering scalable and reliable database services to manage metadata effectively during peak demand, with features such as change data capture and row-level data homing to maintain performance and deliver accurate content to global users.

Choosing the best-priced cloud provider for OLTP workloads among AWS, GCP, and Azure requires a nuanced analysis as no single provider universally excels across all use cases. A study detailed in the 2022 Cloud Report evaluated 56 instances across these platforms, revealing that each offers comparable price-for-performance options. Notably, Google's t2d-standard-32 emerged as a top performer, but all three clouds have instances in the $0.04-$0.05 per TPM range, suggesting an effective tie for cost-efficiency. Larger node sizes generally provide better price-for-performance due to lower storage-to-cost ratios, and general-purpose storage solutions are more economical than high-performance options, although the latter may benefit specific workloads. Despite the overall similarity in pricing, individual instance configurations display variability, emphasizing the importance of choosing based on specific workload needs.

The article by Wade Waldron explores the distinctions and similarities between relational database entities and domain-driven design (DDD) entities. Relational database entities are defined by their unique identifiers and attributes, represented as rows in a table, and are primarily concerned with modeling real-world objects within the constraints of a database schema, including relationships through foreign keys. Conversely, DDD entities prioritize the identity of an object over its attributes, focusing on domain modeling and encapsulating behavior and operations, often represented in class diagrams without specific ties to database implementation. While both types of entities aim to model real-world objects, relational entities are more concrete and implementation-focused, whereas DDD entities are more abstract and domain-focused, leading to potential differences in their application despite some surface-level similarities. The article concludes that understanding these differences is crucial for effectively using the right tools for specific modeling tasks, emphasizing the transition from domain modeling to database implementation.

Form3, a UK-based company, faced the challenge of needing both the speed of a NoSQL database and the consistency of a relational database for their new FPS gateway project. Initially relying on Redis for its speed, Form3 found it lacked the necessary consistency, prompting them to explore alternatives. They opted for CockroachDB, which provided both the speed and consistency required, proving effective for handling over a million transactions per day. To ensure resilience against potential cloud vendor outages, Form3 is implementing a multi-cloud strategy with Kubernetes clusters across Azure, AWS, and GCP, utilizing CockroachDB's multi-cloud capabilities. This approach aims to maintain continuous service availability, even in the event of an AWS outage, and supports seamless migration from their previous Postgres/AWS setup due to CockroachDB's compatibility with the Postgres protocol. Form3's strategy represents a significant step toward building robust, unkillable payment infrastructure.

Mux, a company focused on simplifying video streaming for developers, utilizes CockroachDB for its high-availability and multi-cloud capabilities to prevent outages similar to the one that affected YouTube TV during the 2018 World Cup. By leveraging CockroachDB's distributed nature, Mux ensures its services can withstand node and regional outages without impacting users, which is crucial for live video streaming. Mux's products, including Mux Data, Mux Video, and Mux Player, are designed to handle the complexities of video processing and distribution, allowing developers to focus on delivering quality content. CockroachDB's operational simplicity and support for multi-region deployments play a significant role in managing Mux's video metadata and ensuring resilience and performance across its infrastructure. The company plans to transition more of its systems from Postgres to CockroachDB, attracted by its ease of use and zero RTO/RPO capabilities, while also exploring the development of specialized hardware for video transcoding to stay competitive.

Architecting a scalable payment system is a complex task due to the need for data availability, consistency, and high availability in financial transactions. Traditional methods like manual sharding can handle increased loads but require significant manual effort and complexity, particularly as businesses grow and need to comply with geographical data storage regulations. CockroachDB offers a modern alternative by providing distributed SQL solutions that maintain ACID guarantees while allowing easy horizontal scaling without requiring changes to application logic. This simplifies the management of payment data, facilitates geographic data distribution, and allows seamless integration into SaaS applications and payment provider architectures. CockroachDB's ability to treat multiple nodes as a single logical database makes it a versatile option for businesses looking to future-proof their payment systems against the challenges of scale.

SQL Query Tuning with DB Console provides insights into using CockroachDB's DB Console for optimizing SQL performance. The blog post highlights the importance of observability in understanding and improving database performance, particularly through tools like the ccloud CLI and DB Console, which offer metrics on SQL statements and transactions. The DB Console includes features such as the Metrics, SQL Activity, Statements, and Transactions pages, which allow users to monitor latency, contention, execution statistics, and more. By identifying problematic periods or statements through these metrics, users can take actionable steps to enhance performance. Additionally, the post emphasizes using features like statement fingerprints, transaction fingerprints, and Explain Plans to diagnose issues. The ongoing development of new features, such as a dedicated insights page, aims to further aid users in resolving performance issues. Marylia Gutierrez, the author, brings her expertise in SQL observability to guide users in leveraging these tools effectively.