October 2022 Summaries

The CAKEstack introduces a novel approach to building fully stateful serverless applications by combining CockroachDB Serverless, advanced authentication and authorization platforms, Kubernetes, and event-driven serverless platforms. CockroachDB Serverless provides a scalable database solution capable of "scaling to zero," allowing applications to automatically adjust to workload demands and only charge for consumed compute resources. The stack emphasizes modularity, offering flexibility in choosing components like authentication platforms and serverless frameworks. This architecture supports seamless scaling from an MVP to full-scale operations without requiring extensive upfront infrastructure investment, enabling businesses to accommodate fluctuating workloads efficiently. By leveraging the CAKEstack, developers can focus on user acquisition and application performance, avoiding technical debt and excessive provisioning costs associated with traditional architectures.

After transitioning to streaming in 2008 and experiencing a significant outage, Netflix moved to the cloud, initially adopting AWS and later popularizing Cassandra for global replication needs. However, by 2019, Netflix encountered limitations with Cassandra for specific use cases requiring consistent data transactions, such as their cloud drive service, content delivery network, and Spinnaker platform. Exploring alternatives like AWS Aurora and DynamoDB, Netflix decided on CockroachDB for its ability to meet their requirements of a scalable SQL database with multi-active topology, global consistent secondary indices, global transactions, open-source framework, and SQL support. By 2020, Netflix had deployed CockroachDB, creating over 100 production clusters and 150+ test clusters, primarily within a single region across three availability zones. The largest cluster holds 26.5 terabytes of data. Netflix's tech blog has detailed their diverse use cases for CockroachDB, providing insights into its deployment and management strategies.

Charlie Custer's post provides a comprehensive guide on migrating data into CockroachDB serverless, emphasizing methods for users transitioning from databases like PostgreSQL or MySQL, or using CSV or Avro files. The process involves several key steps, starting with preparing database dump files, followed by importing and converting the schema using CockroachDB's tools. Data importation can be achieved through various methods, including the IMPORT command, AWS DMS, local file importation using the SQL shell, or third-party tools. Users are encouraged to test and update their applications after migration to ensure data integrity and application functionality. The guide also highlights the unique considerations and performance testing required when using distributed SQL databases like CockroachDB, compared to traditional SQL databases.

Cockroach Labs chose Kubernetes for its managed service, CockroachDB Dedicated, due to its robust automation capabilities despite the complexity it introduces. Initially, the team evaluated various orchestration technologies and ultimately found Kubernetes advantageous because it allows software engineering and SRE teams to focus on application-level concerns rather than machine-level details. This decision was influenced by Kubernetes' ability to provide a cross-cloud abstraction, which is important for CockroachDB's deployments on GCP and AWS, and its capacity to automate tasks such as rolling updates and cluster management efficiently. Although Kubernetes adds cognitive load, particularly with its complex networking stack, the benefits of automation and orchestration were deemed to outweigh these challenges. The decision reflects broader industry trends where many users may adopt Kubernetes primarily for its automation primitives rather than for its binpacking capabilities, especially in the context of utilizing elastic compute resources from public clouds.

Devsisters, an international gaming company based in South Korea, achieved massive success with their game Cookie Run: Kingdom, thanks in part to their strategic decision to use CockroachDB for database management. With over ten million downloads and the ability to handle 60,000 transactions per second during peak times, the game imposed far heavier transactional workloads than initially anticipated. Early in the development, Devsisters identified the limitations of Couchbase and opted for CockroachDB, which they found superior in performance and scalability compared to other options like Aurora and DynamoDB. This choice was crucial as it allowed the game to smoothly scale in response to its viral popularity, maintaining uptime and performance even as demand surged far beyond expectations. Devsisters' decision to architect for scale from the outset, combined with the engaging nature of the game, ensured that they could accommodate the influx of players without technical disruptions, underpinning the ongoing success of Cookie Run: Kingdom.

Authored by Mike Bookham and Dan Sheldon, the blog post discusses the automation of database operations using Terraform, particularly focusing on deploying CockroachDB Dedicated with Terraform in combination with AWS EKS. Terraform, a widely adopted Infrastructure as Code (IaC) tool, allows users to define the desired state of their infrastructure, automating processes like cloud infrastructure creation and application deployment. The post provides a step-by-step guide for deploying a CockroachDB Dedicated Cluster using the newly released CockroachDB Cloud Terraform provider, emphasizing the importance of prerequisites such as Git CLI, Terraform, Cockroach Cloud API Key, AWS CLI, and kubectl. It outlines the process from cloning the repository to applying the Terraform script, which sets up the necessary components and validates connectivity between the AWS EKS Kubernetes cluster and CockroachDB. The blog highlights the efficiency and consistency benefits of using IaC in managing infrastructure and anticipates future enhancements to the CockroachDB Cloud Terraform provider to improve the deployment experience further.

Shipt, known for its grocery delivery services, developed a robust distributed payments system to support its operations, including orders on Target's website. The system was designed with multi-region capability from the outset to ensure high availability, using a model where payments function as state machines to manage states like "authorized" and "captured." This approach necessitated careful concurrency control to prevent issues arising from concurrent processes. To address challenges associated with traditional RDBMS setups, such as lag during failovers, Shipt opted for CockroachDB due to its mature, truly distributed nature and the convenience of its managed cloud solution. This decision minimized application complexity by embedding much of the system's intricacy within the database itself, emphasizing the importance of considering data affinity in multi-region applications.

CockroachDB serverless offers a "forever-free" SQL database solution designed to eliminate the complexities of server management by charging users only for the resources they actually utilize. This serverless architecture leverages multi-tenancy, where each tenant can feel like they have their own dedicated CockroachDB cluster, despite sharing the underlying infrastructure with others. The system dynamically scales by allocating resources according to real-time demand, using Kubernetes to manage SQL pods, which are isolated per tenant to ensure security and performance. These pods can be created or destroyed quickly in response to changes in load, keeping costs low and efficiency high. The architecture includes sophisticated security measures, such as isolating tenant data within shared storage nodes using unique identifiers, and ensures fairness in resource distribution through an integrated admission control system. The autoscaler continuously monitors and adjusts the number of SQL pods based on recent CPU usage, maintaining a balance between stability and responsiveness. CockroachDB's approach allows for rapid deployment and cost-effective scaling, making it an attractive option for developers looking for a robust, scalable, and free database solution.

CockroachDB has launched a serverless version of its database, offering features like change data capture, backup and restore, alongside a 99.99% uptime SLA, for free. This serverless model allows users to avoid upfront server estimations by paying only for the requests and storage they use, with automatic scaling to accommodate varying loads. The architecture supports multi-tenancy by isolating SQL processes per tenant while sharing the key-value storage layer, ensuring security and efficient data management for multiple tenants simultaneously. The system uses Kubernetes to manage serverless clusters, allowing quick creation and cost-effective operation by hosting multiple virtualized clusters on a single physical cluster. The autoscaler dynamically allocates resources based on traffic, maintaining efficiency and performance, while proxy pods manage connections and scaling. CockroachDB Serverless aims to attract users with its flexible, cost-effective infrastructure, hoping successful apps will transition to paid services.

The blog post, authored by Charlie Custer, provides insights from the 2022 Cloud Report on the performance of various Google Cloud Platform (GCP) instances for Online Transaction Processing (OLTP) workloads. It details the results of tests conducted on 18 different GCP instance types across more than 3,000 runs, focusing on overall median transactions per minute (TPM), median TPM per virtual CPU (vCPU), and price-for-performance using a Cockroach Labs TPC-C Derivative Nowait metric. The t2d instances, featuring AMD Milan processors, consistently emerged as top performers in various categories, including overall performance, per-vCPU performance, and cost efficiency, often outperforming larger nodes and high-performance storage configurations. The report also offers comparisons with similar instances on AWS and Azure, providing comprehensive benchmarks for CPU, storage, and networking performance, and is available for free to assist GCP customers in selecting the optimal instance for their OLTP workloads.

CockroachDB Serverless offers a truly serverless database experience by allowing automatic scaling of resources based on workload, ensuring that users only pay for what they use, and potentially allowing them to use the service for free indefinitely. Unlike some serverless databases that do not scale down to zero, CockroachDB Serverless guarantees scaling to zero, meaning no charges occur when there is no activity. It provides a free initial tier with monthly credits for requests and storage, and users can set spend limits to avoid unexpected charges while receiving alerts as they approach these limits. The service is designed to handle surges with "burst capacity" and offers tools for managing and optimizing serverless workloads, making it an attractive option for developers seeking cost-effective and efficient database solutions.

Distributed SQL represents an evolution in database technology, offering a scalable, resilient, and globally distributed solution that meets the demands of cloud environments while maintaining the reliability of traditional relational databases. As organizations move to the cloud, they find legacy databases lacking in scalability and global distribution, prompting a shift to distributed SQL solutions like Google Cloud Spanner, Amazon Aurora, Yugabyte, FaunaDB, and CockroachDB. These databases provide key features such as seamless scaling, transactional consistency, geo-replication, SQL language support, and the ability to operate across multiple cloud environments, addressing challenges like data locality and multi-cloud deployment. Foundational requirements for these databases include administration, optimization, security, and integration, ensuring they meet enterprise needs and can handle both mundane and mission-critical workloads. As the category matures, the focus is on implementing these advanced features effectively in production, marking distributed SQL as a significant advancement in managing data in the cloud.

The blog post, written by Dan Kelly, explores the practical applications and benefits of serverless architecture using CockroachDB serverless, highlighting two real-world examples: Rainmaker Games and TIMU. Rainmaker Games, a gaming discovery and community site, leverages the serverless stack to simplify operations, eliminate manual tasks, and focus on enhancing user experience by connecting gamers and facilitating smarter gaming investments. TIMU, a collaboration platform in beta, combines features similar to Google Meet, Slack, Trello, and Google Drive, benefiting from serverless capabilities to efficiently manage scaling and reduce operational costs. The post emphasizes the seamless scalability and reduced maintenance burdens of serverless architecture, illustrating how it allows applications to grow and adapt without significant overhead.