March 2022 Summaries

This post discusses the concept of Write Skew and Snapshot Isolaation in database transactions. It explains that while Snapshot Isolation is often considered a good balance between performance and correctness, its precise meaning can be unclear. The text delves into the differences between Serializable and Anomaly SERIALIZABLE (SNAPSHOT) isolation levels, highlighting that SNAPSHOT allows for outcomes not possible in either serial execution. It also explores how Snapshot Isolation can lead to non-serializable behavior and what must be done to eliminate these anomalies. The post concludes by emphasizing the importance of understanding Snapshot Isolation given its prevalence in mainstream databases.

Schema migration tools like Flyway and Liquibase are essential for managing database schema modifications. These tools provide version control, automation, easy rollback of changes, and the ability to reproduce schema changes across multiple environments. CockroachDB has built official support for both Flyway and Liquibase, making it easier for developers to manage their database schemas. By using these tools, developers can increase efficiency and reduce errors during schema migrations.

CockroachDB, a distributed SQL database, has developed a method for running serverless databases on Kubernetes. The architecture separates the system into two layers: SQL and Key/Value (KV) storage. In a dedicated deployment, these layers communicate in the same process; however, in a multi-tenant deployment, each layer is split into its own process. This results in one large stateful cluster of storage pods that act as the back end for many stateless SQL pods. The KV Cluster is essentially a dedicated CockroachDB deployment. The SQL pods are where the uncharted territory lies. These pods, like the storage ones, are still CockroachDB but with a slightly different entry point. Thanks to the architecture of CockroachDB's SQL execution layer, SQL pods are horizontally scalable. This means that the quality of service tenants experience can be controlled by adjusting resource limits and requests on SQL pods or modifying the number of replicas allocated to them. The low latency resumption process allows for SQL Pods to spin up in an un-bound state, which are then bound to a tenant when needed. This enables CockroachDB Serverless to avoid pod start-up times and provide fast startup times. The autoscaler component within the operator decides the number of SQL pods that each tenant needs at any point in time by monitoring CPU load across all SQL pods for each tenant in the Kubernetes cluster. The CrdbTenant controller updates the actual state to the desired state at a controlled rate, ensuring the desired number of SQL pods are allocated. The draining process prevents constant churning of unstamped pods and allows for more efficient resource usage by tenants. The SQL Proxy component is what end users connect to when they use CockroachDB Serverless. It routes connections to corresponding stamped SQL pods, and if there are no active SQL pods due to scaling down, the low-latency resumption process comes into play. The user can get started with a free CockroachDB Serverless account by signing up at https://cockroachlabs.cloud/signup.

Early startups often face the challenge of technical debt due to quick, "right enough for now" solutions in their initial stages. As they grow and experience increased user demand, these decisions can become limiting factors. To avoid this, startups should consider adopting distributed architecture from the beginning or as soon as possible. Distributed systems are more complex initially but offer scalability benefits that can support growth. Platform-as-a-Service (PaaS) and Infrastructure-as-a-Service (IaaS) solutions can help manage this complexity while providing the advantages of distributed architecture.

Aika Zikibayeva is a Partner Marketing Manager at Cockroach Labs, where she collaborates with both the Product Marketing and Channels & Alliances teams. Her career journey began in foreign affairs before transitioning to marketing. She chose to work at Cockroach Labs due to its innovative technology, passionate values, and commitment to work-life balance. Zikibayeva is excited about building the foundation of how Cockroach Labs positions itself within its ecosystem. Her advice for those starting careers in product & partner marketing includes developing diplomacy skills, aligning teams with overall goals, and building trust with colleagues. The Product Marketing team at Cockroach Labs is described as small but mighty due to their valuable information and ability to empower other teams.

Avoiding tech debt is crucial for startups to scale effectively. To minimize or eliminate tech debt, startups can follow these six strategies: don't take on unnecessary debt; minimize ops work by opting for managed services; stay flexible in design and tool choices; avoid solving problems that are already solved; establish coding best practices early; and balance present needs with future goals. By making smart decisions in their tech stack, startups can build for scale without taking on significant tech debt.

Test Engineering (TestEng) at Cockroach Labs is a team of accomplished engineers responsible for driving the quality of CockroachDB through automated testing and program analysis. The team collaborates with other departments to establish quality gates, test strategies, and best practices. TestEng owns correctness and performance validation methodologies as well as the test infrastructure. Joining the Test Engineering team offers opportunities to shape not only the organization but also one's career path, working closely with talented engineers on challenging problems related to infrastructure, exploratory testing, and performance.

Bitski is a startup company based in San Francisco that has built an NFT marketplace platform with Swiss bank-level security and an innovative approach to processing blockchain transactions. The architecture of their high functioning NFT marketplace emphasizes speed, consistency, and availability. Their Blockchain Transaction Operator queues up transactions in batches, gathers all the information each transaction is going to need, and then broadcasts that batch of transactions to the blockchain, allowing for extremely fast processing times. Bitski's NFT marketplace built on CockroachDB has proven to be an order of magnitude faster than what everyone else is doing.

Application architecture is crucial for startups, as getting it right from the beginning can save significant headaches down the line. A typical startup application architecture consists of a user-facing front end, back end instances connected via a load balancer, and data passed horizontally to a data warehouse or vertically to a relational database. For high availability, modern applications often use active-active databases like CockroachDB, which distribute reads and writes across multiple nodes for better performance and resilience. When scaling to multiple regions, CockroachDB allows for multi-region deployments to be treated as a single logical database, ensuring optimal performance and availability. Building with distributed SQL databases from the outset can make it easier and more affordable to architect applications for scale, even if scaling is not immediately necessary.

This tutorial demonstrates a multi-region deployment of CockroachDB across two Azure regions using Kubernetes and virtual machines. The deployment includes the following steps: 1. Infrastructure setup: Create three resource groups, one for each region, and deploy three virtual machines in each region. 2. Network setup: Configure VNet peering between the two regions to enable communication between them. 3. Kubernetes cluster setup: Deploy k3s Kubernetes clusters on the virtual machines using k3sup utility. 4. CockroachDB deployment: Use a Python script and modified Kubernetes manifests to deploy CockroachDB across the two regions. 5. Virtual machine deployment: Add the third region by manually installing CockroachDB on three virtual machines in that region. The final setup includes nine nodes distributed across three regions, with six nodes running as Kubernetes pods and three nodes running natively on virtual machines. The multi-region deployment ensures high availability and fault tolerance for the CockroachDB cluster.

The article discusses the development of a highly available spatial data demo using CockroachDB v20.2. This single-page web app showcases a tourist on a map with nearby amenities, such as pubs and cafes, updated in real-time as the user navigates the map. The demo utilizes Leaflet, Python, Flask, Mapbox, CockroachDB K8s operator, and data from OpenStreetMap. The author emphasizes the importance of highly available spatial data to avoid situations where GPS applications fail, leaving users stranded. A video and GitHub repository are provided for further exploration of this demo.

This tutorial guides developers through creating and deploying an AWS Lambda function that uses Python and connects to a CockroachDB Serverless database. The process involves setting up an AWS account, creating an IAM user, installing the AWS CLI, creating a CockroachDB Serverless cluster, cloning the Python function, creating a deployment package, configuring AWS, deploying the function to AWS Lambda, and invoking the function. The tutorial also provides optional steps for using the CockroachDB SQL shell to view results.

Cockroach Labs uses rubrics in their hiring process to ensure more equitable and data-driven decisions. Rubrics are tools that allow for objective comparison of candidates' qualifications based on established job-related criteria, reducing potential bias. The company developed a set of standard, company-wide measures mapped to its mission and values, as well as department-level rubrics with specific criteria relevant to each team. They also emphasize the importance of considering "culture add" qualities rather than just "culture fit," which can hinder diversity. Inclusive rubric essentials include building a scoring standard, working with hiring teams, and leaving room for potential. By combining core criteria with additional room for advocacy in areas such as culture add and potential, Cockroach Labs aims to evaluate candidates on key strengths while also championing growth for new employees and their own team.

In this tutorial, we built a Jamstack web application using JavaScript as the primary language. We used React for our frontend and CockroachDB Serverless to store data in a PostgreSQL-compatible database. Additionally, we utilized Netlify Functions to manage the REST API layer of our application. To build this application, we followed these steps: 1. Set up the development environment by installing Node.js and npm on our computer. 2. Initialized a new project using create-react-app and installed necessary dependencies. 3. Created a CockroachDB Serverless cluster to store data for our application. 4. Implemented Netlify Functions to interact with the database through REST API endpoints. 5. Styled the application using CSS and styled components. 6. Defined routes to pages that use custom styles. 7. Created a default route and a create-activity route in static-config.js. 8. Implemented the CreateActivity page with input fields for users to type in activity details. 9. Implemented the default activities page to fetch all created activities using the GET /activities Netlify Function. 10. Deployed the entire application to Netlify using the netlify CLI. By following this tutorial, you have built a Jamstack web application that uses serverless technology for data storage and retrieval. You can now expand this application with more features or use the provided code as a starting point for your own projects.

Managing multiple Kubernetes clusters across different environments can be complex, but Amazon EKS-Anywhere allows DevOps teams to run Kubernetes outside of AWS, including on virtual and bare metal machines. CockroachDB is a cloud-native, distributed SQL database designed for running within a Kubernetes cluster, providing consistent control and simplified deployment of databases. Deploying CockroachDB within an EKS Anywhere Kubernetes cluster offers security, storage configuration, elastic scale, and performance upgrades on-premise. Companies can achieve similar performance and experience running CockroachDB on EKS Anywhere as they do running EKS in AWS.

Virtual Private Cloud (VPC) peering is a networking connection between two VPCs that enables private IP address communication. It offers improved security, cost savings on network transit costs, and flexibility for services not requiring public Internet access. CockroachDB Dedicated allows users to set up VPC peering with GCP or AWS PrivateLink connections. This enhances user experience by eliminating the need for allowlisting IP addresses within CockroachDB Dedicated before connecting.

In multi-region applications, data homing is crucial for delivering low latency experiences to a global user base. Traditionally, developers had to write complex code to manage data placement across regions. CockroachDB's REGIONAL BY ROW feature simplifies this process by allowing automatic data homing based on the crdb_region column. This enables applications to maintain the simplicity of single-region deployment while benefiting from multi-region performance. Experiments show that a default REGIONAL BY ROW table performs similarly to manually partitioned tables, and auto-rehoming can further improve application performance under certain conditions.