May 2022 Summaries

CockroachDB's latest paper, presented at SIGMOD 2022, details advancements in multi-region database deployments, emphasizing a new Transaction Management protocol that enhances local, strongly consistent reads from any database replica. The paper showcases how CockroachDB simplifies configuration for multi-region applications by using a high-level declarative SQL syntax that translates data access locality and availability goals into practical database configurations, including replica placement and data partitioning. This innovation allows for linear scalability with the number of regions and significantly reduces tail latency, demonstrating improved performance for diverse workloads. The paper, a collaborative effort across various engineering domains at Cockroach Labs, will be presented by Nathan Vanbenschoten, with additional talks by the company's Senior Director of Engineering, Adam Storm, providing insights into the technology and culture of Cockroach Labs. The team also looks forward to participating in VLDB 2022 to further showcase these capabilities.

CockroachDB 22.1 introduces a REST API that allows users to automate database operations by managing clusters programmatically instead of through the web interface. To use the CockroachDB Cloud API, users must obtain a secret key via a CockroachDB cloud account, which uses bearer token authentication. With the API, users can list clusters, create new clusters with specified parameters, retrieve detailed cluster and node information, adjust serverless spend limits, and delete clusters. Additionally, the API allows for the creation and deletion of SQL users. These operations are executed through specific API requests, each requiring the user's unique secret key and relevant cluster identifiers. The API is capable of performing a wide range of operations, as detailed in its comprehensive documentation.

CockroachDB 22.1 is a robust database solution designed to streamline application development from prototyping to large-scale deployment, addressing common challenges like bottlenecks and complexity associated with other platforms. This release introduces enhancements across multiple stages of development, such as a new command-line interface for quicker setup, support for Node.js and Typescript via Prisma, and features like time-to-live for data management. It also offers improved automation for cluster management, troubleshooting, and backup efficiency, alongside integrations with tools like Datadog for better monitoring. For scaling, CockroachDB 22.1 introduces Super Regions to manage data domiciling and features like Quality of Service and Admission Control to maintain performance during peak demands, ensuring seamless operation across geographies.

CockroachDB's new feature, Admission Control, introduced in version 22.1, effectively addresses CPU imbalance issues by optimizing CPU utilization across nodes in a cluster, as demonstrated by a successful upgrade from version 21.1.11. The feature was tested on a workload involving 200 different queries, where initially one node operated at near 100% CPU, leading to performance issues. After a zero-downtime upgrade and activation of Admission Control via a few SQL commands, there was a noticeable improvement in CPU utilization, with all three nodes operating within a 3% range of one another, showcasing the feature's efficacy. This improvement highlights the innovative and thoughtful development by Cockroach Labs, encouraging users to explore the new functionalities offered in the latest release to enhance performance and stability.

Deploying multi-region applications using Google Cloud Run and CockroachDB offers enhanced resiliency, reduced latency, and simplified data localization, making it an attractive architecture for developers aiming to optimize user experience across diverse geographic areas. Google Cloud Run allows developers to run containerized applications with effortless scaling and integration, while CockroachDB provides a distributed SQL database that automatically manages data across multiple nodes and regions, ensuring data consistency and availability even in case of regional failures. The architecture supports efficient routing of requests to the nearest Cloud Run service using a global External HTTPS Load Balancer with Anycast IP addressing, thus minimizing response times. Though inter-regional latency can impact performance, strategic data placement and the ability to adjust primary regions within CockroachDB help in maintaining optimal query speeds. This setup not only simplifies the traditionally complex task of building distributed systems but also supports serving applications and data closer to users, thereby enhancing application speed and reliability.

CockroachDB, an open-source distributed SQL database, is designed to enable seamless scalability and high availability for businesses without the need for complex re-architectures. It operates with a shared-nothing architecture and employs the Raft consensus protocol to ensure data consistency and resilience against failures. The database can be deployed as the central OLTP component of an application's data architecture, evolving from a single-node proof-of-concept to a globally scaled service. Its geo-partitioning feature allows for fine-grained control over data replication, addressing issues of latency and data sovereignty essential for compliance with regulations like the EU's GDPR. CockroachDB's architecture facilitates zero-downtime upgrades and minimizes disruptions, making it suitable for enterprises that require rigorous service level agreements. Through an interactive D3 simulation, users can visualize the deployment stages and understand how CockroachDB supports the growth of businesses from local to global scales.

The article "No Time to Live: James Bond Explains Row-Level TTL" by Dan Kelly introduces row-level Time-To-Live (TTL) as a significant feature in CockroachDB 22.1, which automates the deletion of expired data. This mechanism, highly requested since 2017, removes the need for developers to write custom scripts for data expiration, simplifying compliance with data retention regulations and improving database management. The feature works by selecting and deleting expired rows through background jobs that are rate-limited to minimize impact on active transactions. Row-level TTL offers a more efficient alternative to traditional methods like table partitioning and supports various use cases, such as managing inactive data and maintaining compliance. The author highlights the ease and efficiency this feature brings to developers and notes that it is part of a broader set of enhancements in the CockroachDB 22.1 release.

KubeCrash is a virtual event coinciding with KubeCon EU 2022, designed to accommodate individuals in the Americas who are unable to attend the conference in Valencia. This initiative presents a series of live, half-day crash courses focused on cloud-native technologies and is the result of collaboration between five open-source companies. The event provides participants with access to workshops and talks led by maintainers of prominent open-source projects like Linkerd, cert-manager, CockroachDB, Pulumi, and Fairwinds' Polaris and Goldilocks, covering topics such as zero-trust identity management, multi-cluster failover, workload optimization, serverless architecture, and multi-cloud engineering. KubeCrash aims to deliver KubeCon-grade content without vendor pitches, offering actionable insights for developers, reliability engineers, cloud security specialists, cloud architects, and platform engineers. The event is scheduled for May 17 and will be hosted and moderated by CNCF Community Ambassadors, ensuring a comprehensive learning experience for attendees.

Event-driven systems provide a dynamic alternative to batch-driven processes by enabling real-time reactions to events as they occur, but they bring challenges such as handling duplicate and out-of-order events. Duplicate events can be managed through idempotency, ensuring that repeated operations do not alter the final outcome, as demonstrated through examples like data structures and banking transactions. To achieve idempotency, unique identifiers such as UUIDs or transaction IDs can be embedded in events, allowing systems to deduplicate effectively without a central bottleneck. Out-of-order events present another challenge, where absolute ordering is impractical due to scalability concerns, and solutions include allowing unordered processing or applying limited scope ordering within specific contexts, like individual bank accounts. These methods help maintain system efficiency and scalability, emphasizing that no universal solution exists but rather a need for tailored approaches based on specific use cases. For further understanding, resources like the Cockroach University course on Event-Driven Architecture offer detailed insights into these systems.

Cockroach Labs has been recognized on Inc. magazine's 2022 list of Best Workplaces for the second consecutive year, a testament to its growth and commitment to fostering an exceptional work environment. The accolade reflects the company's successful adaptation to changing workplace norms, particularly during the pandemic, by implementing a new distributed workplace model that balances flexibility with productivity and collaboration. As offices began reopening, Cockroach Labs introduced a hybrid working environment, establishing a multi-use headquarters in New York and investing in offices in Toronto and San Francisco. The company continues to challenge conventional workplace practices by emphasizing diversity, innovation, and inclusivity, with a focus on inspiring employees to excel, whether working remotely or in the office. Cockroach Labs is actively recruiting for various roles to further its mission of becoming a leading database company.

Cockroach Labs celebrates the release of "CockroachDB: The Definitive Guide," co-authored by database expert Guy Harrison, Cockroach Labs co-founder Ben Darnell, and head of education Jesse Seldess, as a comprehensive resource aimed at making distributed SQL more accessible. Guy Harrison, a seasoned database professional, shares his journey from working with pre-relational databases to the current evolution of distributed SQL systems, highlighting CockroachDB's role in merging the benefits of relational databases with distributed database innovations. He expresses surprise at CockroachDB's feature completeness despite its relative newness and emphasizes the importance of specialized databases tailored for specific workloads, such as operational databases like CockroachDB and analytical ones like Snowflake, while noting the ongoing significance of graph databases. Currently, Harrison is exploring a project utilizing CockroachDB and IPFS, reflecting his ongoing engagement with cutting-edge database technologies.

Deploying a Netlify application with TypeScript and CockroachDB involves setting up a scalable, serverless database system that's both cost-effective and straightforward to implement. The process begins with creating free accounts on Netlify and CockroachDB, followed by deploying a sample application using TypeScript. The application uses CockroachDB, a cloud-native distributed SQL database, integrated via Prisma for database management. After obtaining the sample application code from GitHub, users set up their databases by modifying environment variables and utilizing Prisma to synchronize the database schema. The deployment includes local testing with Netlify's CLI and concludes with a full public deployment, all of which can be achieved without incurring any costs. This approach demonstrates how modern tools can facilitate the development and deployment of scalable applications efficiently and inexpensively.

The article explores the limitations of the CAP theorem in distributed systems, emphasizing how CockroachDB, a CP system prioritizing consistency over availability, navigates these constraints to achieve high availability. It argues that while the CAP theorem demands a trade-off between consistency and availability during network partitions, real-world scenarios often allow for systems like CockroachDB to remain highly available through strategic engineering practices. By maintaining strong consistency, CockroachDB ensures reliable data handling and minimizes complexity in application code, even though it might incur higher latency in some cases. The article highlights the introduction of bounded staleness reads in CockroachDB, which offers a balance between consistency and availability by allowing minimally stale reads during network partitions. Overall, the strong consistency foundation of CockroachDB provides developers with a reliable and straightforward data management experience, making it a preferred choice for many applications despite the potential trade-offs in speed.

The article provides a comprehensive guide on building and deploying a simple Django web application using CockroachDB as the database, specifically through CockroachDB Serverless. The application developed is a game leaderboard that allows users to input gameplay data, which is stored and retrieved from the database to display various game statistics. The process involves setting up a CockroachDB account, installing necessary dependencies like Django and psycopg2, configuring the database connection using environment variables, and implementing Django models, views, and templates to handle data interactions. Additionally, the guide covers deploying the application to Heroku, including configuring static files with Whitenoise and using Git for version control. The tutorial emphasizes the seamless integration between Django and CockroachDB, highlighting the scalability benefits of using a serverless database, and encourages developers to expand on the foundational code provided to build more complex applications.

Choosing the right indexes for a database involves adhering to three basic rules: usefulness, clustering, and scattering. Usefulness ensures that indexes speed up queries and enforce constraints, although they can slow down write operations. Clustering involves organizing records accessed together to minimize network operations, particularly important in distributed databases like CockroachDB. Scattering, however, keeps unrelated records apart to avoid hotspots and improve scalability. When selecting unique IDs, options include sequences, timestamps, and randomness, each with its trade-offs in terms of performance and scalability. Hash-sharded indexes are a unique feature in CockroachDB designed to handle sequential traffic by distributing it uniformly, thus avoiding single-range hotspots. They offer improved write performance for sequentially-keyed indexes while slightly affecting read performance, and can be created by enabling specific session variables and using SQL extensions.

CockroachDB, a database platform known for managing transactional data globally with high availability and multi-cloud capabilities, has partnered with Satori to enhance data security features. This collaboration aims to address the increasing need for advanced data security across various business scales and types of data stores by integrating Satori's DataSecOps platform. Satori provides features like fine-grained access control, data masking, and anonymization, as well as scalable data access auditing, which complement CockroachDB's existing security measures such as Identity and Access Management, data protection, network security, and compliance. The integration works by placing Satori's Data Access Controller between users and the CockroachDB cluster, allowing for seamless application of security controls without altering the database. Depending on their needs, customers can choose different deployment patterns, such as Private SaaS or multi-tenant offerings, to fit their specific requirements. The setup process is straightforward and can be completed with minimal downtime, making it accessible for organizations looking to enhance their data security with CockroachDB and Satori.