June 2022 Summaries

Over the past two years, coding livestreams have gained popularity as developers shared their projects and problem-solving processes online, creating communities in the process. For beginners looking to learn programming, several streamers offer accessible content, including Leon Noel's Web Dev Bootcamp on Twitch, which aims to help those affected by the pandemic and underrepresented groups learn full-stack web development. Additionally, Pachi Carlson learns Python live, Prince Wilson explores Rust through Exercism.io, and the creator of Coding Garden provides full-stack app tutorials with a focus on JavaScript. Other notable streams include Jeseekia Vaughn's 100 Days of Code challenge, Mayuko's co-working sessions for coding sprints, and Aydrian Howard's casual coding streams on building Jamstack applications, enhanced by occasional appearances from his corgi.

In the 2022 Cloud Report, AMD's Milan processors emerged as the top performers for OLTP workloads across leading public cloud providers AWS, GCP, and Azure, demonstrating superior performance and cost-efficiency compared to Intel's offerings. The report utilized a variation of the TPC-C benchmark to assess OLTP performance and CoreMark for direct CPU performance metrics, highlighting that AMD Milan processors consistently outperformed, securing the top spots in both performance and price-for-performance categories. Despite AMD's dominance, Intel's latest-generation processors, particularly the Cascade Lake series, remained highly competitive, emphasizing that the choice of CPU may vary based on specific workload requirements. The report provides extensive analysis and insights across 56 different instance types, including considerations for storage and network performance, offering readers a comprehensive guide to selecting the optimal instance type for their needs.

The blog post details the process of building a multi-region Python application using Django, CockroachDB, and Kubernetes, demonstrating how to leverage CockroachDB's multi-region capabilities. The author, a newcomer to Django, explains the framework's benefits for rapid development and its application in a multi-region context. The process involves updating a basic Django application to include data domiciling capabilities, allowing specific datasets to be stored in designated geographic regions to improve performance and comply with data sovereignty regulations. This involves configuring a CockroachDB cluster to use regional data placement and adjusting the Django application to handle an additional 'cloud' field for data locality. The post emphasizes the ease of developing such applications using Django and CockroachDB, highlighting performance enhancements and compliance advantages associated with data domiciling.

SELECT FOR UPDATE is a SQL command designed to manage concurrent transactional workloads by locking rows identified in a SELECT query until the transaction is committed, thereby preventing transaction conflicts and reducing unnecessary retries. It enhances efficiency by placing concurrent transactions in a queue to be executed sequentially, ensuring smoother processing of contended transactions. However, its implementation and support vary across different SQL databases, with some like MySQL supporting it and others like SQLite opting for different transaction handling methods due to their approach to isolation levels. In databases like CockroachDB, SELECT FOR UPDATE can be used alongside serializable isolation without locking the entire database, supporting performance and consistency simultaneously. Various parameters such as NOWAIT and FOR SHARE modify its behavior, offering different levels of row-locking, but their availability also depends on the specific database system.

Building a pain-free metadata store involves a series of best practices, beginning with understanding the nature of metadata, which can be broadly categorized into forensic and control data, with a focus on the latter due to its mission-critical nature and relational complexity. The process starts with selecting the right database tailored to specific needs rather than convenience, as illustrated by Rubrik's experience transitioning from Cassandra to CockroachDB for better consistency and performance. Thoughtful database schema design is crucial, requiring careful planning to avoid major adjustments post-deployment, and incorporating a "human schema" is essential to ensure that all team members, regardless of technical expertise, can access and understand the data through resources like a data dictionary. Schema changes should be additive to facilitate integration with existing application services and avoid disruptions, necessitating strategies for backfilling data when new columns are added. Finally, while migrations might eventually become necessary due to the evolving needs of an application, thorough upfront planning and adherence to these best practices can delay and mitigate the challenges associated with such transitions.

Financial technology (fintech) companies are vital in today's world, providing essential services that require scaling efficiently, reliably, and globally. Nubank, a digital bank originating in Brazil, leveraged CockroachDB to automate scaling and simplify operations, enabling rapid growth into multiple countries. Form3, a UK-based payment platform, addressed reliability concerns by implementing a multi-cloud strategy with CockroachDB to ensure continuous service even if a cloud provider experiences downtime. Pismo, operating in Brazil and the UK, needed a solution for global data compliance and achieved this by deploying CockroachDB Dedicated on Kubernetes, allowing for horizontal scalability and control over data locality. These examples highlight the critical role of CockroachDB in enabling fintech companies to handle enormous transaction volumes while maintaining compliance with regional data regulations, ensuring dependable service amidst increasing consumer reliance on digital financial services.

The blog post discusses how five fintech companies—Aurora Payments, Fi, Pismo, Form3, and Birchal—across different continents have successfully implemented CockroachDB to overcome challenges related to scalability, availability, and consistency in their payment processing systems. Aurora Payments in the U.S. transitioned from PostgreSQL to CockroachDB to manage a vast amount of private and transactional data, ensuring high availability and redundancy. Fi in India chose CockroachDB for its ability to scale across regions and comply with complex data regulations, supporting their rapidly growing customer base. Pismo, operating in Brazil, uses CockroachDB to modernize its banking platform and expand globally while maintaining regulatory compliance. The UK-based Form3 utilizes CockroachDB for its multi-cloud, API-first platform to ensure continuous operation even if a cloud vendor faces an outage. Lastly, Birchal in Australia adopted CockroachDB for its ability to handle transaction-oriented workloads efficiently, providing a seamless user experience for their equity crowdfunding platform. These companies demonstrate CockroachDB's capability to support fintech operations without sacrificing performance or consistency.

The blog post explores the concepts of Recovery Point Objective (RPO) and Recovery Time Objective (RTO), which are critical metrics for IT resilience determining acceptable data loss and downtime, respectively, during system outages. It highlights the financial impact of IT downtime, driving companies to pursue near-zero RPO and RTO solutions. Achieving these objectives is complex, involving a multi-layered approach with database systems, clustering, and replication technologies. The post emphasizes the challenges and limitations of current methods, like active-active and NoSQL systems, and introduces CockroachDB as a solution offering zero RPO with built-in fault tolerance and ACID guarantees, reducing complexity and cost. It advocates for IT leaders to adopt advanced database technologies like CockroachDB to make zero RPO a standard for core business applications, highlighting its benefits in maintaining data consistency, minimizing downtime, and enabling innovation without the trade-offs traditionally associated with data protection.

Cockroach Labs has integrated the OpenTelemetry Collector into its cloud infrastructure to enhance observability for its CockroachDB Dedicated service, allowing flexible third-party monitoring integrations. This open-source framework, backed by the Cloud Native Computing Foundation, facilitates the collection, transformation, and export of telemetry data such as metrics, logs, and traces. The modular design of the OpenTelemetry Collector includes components like receivers, processors, and exporters, which can be configured into pipelines to support various telemetry formats and export targets, starting with Datadog integration. Deployed in Kubernetes, the Collector uses an agent and gateway model to manage telemetry data efficiently with minimal impact on system resources. Cockroach Labs aims to expand the use of the Collector beyond metrics to include traces, logs, and internal telemetry, aligning with future observability needs.

CockroachDB 22.1 introduces the concept of super regions, a feature designed to simplify data domiciling while ensuring regional resilience and compliance with regulations like GDPR. Super regions allow data to remain within specified geographic boundaries, even during regional failures, by grouping multiple cloud regions into larger areas. This capability addresses the challenge of data domiciling, which previously required complex logic and multiple database copies, by enabling straightforward deployment across regions with minimal changes to applications. Super regions particularly benefit multinational businesses and heavily regulated industries, as they help comply with strict data localization laws. Additionally, CockroachDB's new row-level TTL feature complements super regions by automating data deletion processes, further aiding compliance with data management requirements. With super regions still in preview, they are available in CockroachDB self-hosted enterprise and CockroachDB Dedicated, making it easier for organizations to manage data across distributed networks while maintaining compliance and resilience.

The 2022 Cloud Report offers a comprehensive analysis of cloud performance, featuring results from over 3,000 OLTP test runs across 56 instance types on AWS, GCP, and Azure, highlighting AMD's Milan CPUs outperforming Intel for the first time in terms of both performance and cost-effectiveness. This year's report, which is more extensive than previous editions, includes testing of a wider range of instance configurations, node sizes, and increased test durations, aiming to provide a deeper understanding of performance variations with workload complexities. The report introduces the Cockroach Labs Derivative TPC-C nowait benchmark, which modifies the traditional TPC-C benchmark to better compare instance types by removing wait times, thus focusing on pure transactional throughput. The 2022 edition emphasizes realism in simulation and provides detailed insights into various cloud configurations, remaining a free resource for those interested in cloud performance metrics.

Jeff Carlson's article delves into integrating CockroachDB with Google Pub/Sub to streamline data streaming across cloud environments. The guide provides a detailed step-by-step process, starting from setting up a CockroachDB Dedicated cluster to creating a Google Cloud Storage bucket and configuring a Google Cloud Platform account. It highlights the benefits of using Google Pub/Sub, which unlocks a suite of Google Cloud tools, offering scalable and secure messaging systems. The integration leverages CockroachDB's Change Data Capture (CDC) capabilities, enabling efficient data streaming for downstream processing, which simplifies IT infrastructure and enhances the ability to manage distributed data systems. By demonstrating how to create a dataflow job that streams data from CockroachDB to Google Cloud Storage, the article illustrates how enterprises can benefit from a simplified architecture that quickly updates downstream systems.

CockroachDB has enhanced its admission control and resource allocation mechanisms to better manage database performance under load, particularly in multi-node transactional databases. The improvements involve hybrid token and slot-based allocation systems that dynamically adjust for CPU and storage resources, along with mechanisms for multi-tenant performance isolation and priority adjustment to minimize priority inversion. CockroachDB addresses challenges like CPU and storage bottlenecks by intercepting CPU-bound work and implementing dynamic slot and token calculation, while an epoch-LIFO queueing system helps manage transaction deadlines. These enhancements shift queuing from the CPU scheduler to admission control queues, maintaining high CPU utilization and reducing latency for high-priority tasks. The system also supports fair resource allocation among tenants and adapts queueing strategies based on observed delays, improving throughput and preventing node failures during overload.

The blog series by Rubrik explores their decision to transition from using Cassandra to CockroachDB as their metadata store for Rubrik CDM, highlighting the challenges faced with Cassandra such as uneven data distribution, inefficiencies of secondary indexes, out-of-memory crashes, and issues with deleted row resurrection. Seeking a solution that could offer high consistency, good performance under stress, and ease of maintenance, Rubrik developed a stateless translator called CQLProxy to facilitate the migration without altering application layers. The adoption of CockroachDB reduced operational overhead and simplified application logic, though challenges like clock skew and backpressure required innovative solutions like the development of Kronos, a custom time service, and additional application and database adjustments. Rubrik's experiences underscore the importance of choosing a metadata store that aligns with system architecture and operational requirements, offering insights into metadata management and distributed database optimization.

In the CockroachDB 22.1 release, the ccloud CLI tool has been introduced to streamline the creation and management of CockroachDB clusters directly from the command line, offering a faster alternative to graphical user interfaces. To get started with ccloud, users can download and install the tool through several methods, including Homebrew for macOS or direct binary downloads for different operating systems like Linux and Windows. Once installed, users can quickly set up their first cluster using the command `ccloud quickstart`, following simple on-screen prompts to complete the process. This new tool represents a small portion of the enhancements in version 22.1, which also includes a new cloud API, Prisma support, and row-level TTL features, among others.