December 2024 Summaries

Cockroach Labs had a remarkable 2024, marked by significant growth, product innovations, and global expansion. The company grew to over 500 employees, moved its offices to larger spaces, and took its annual customer conference, RoachFest, global for the first time. On the product side, CockroachDB introduced various under-the-cover performance improvements, expanded its MOLT (Migrate Off Legacy Technology) tools, and delivered physical cluster replication (PCR) and logical data replication (LDR) for self-hosted customers. The company also launched its first-ever State of Resilience Report, which revealed the true cost of downtime across industries. CockroachDB's vector search capabilities compatible with PostgreSQL's `pgvector` extension were introduced to enhance functionalities such as semantic search, recommendation systems, and natural language processing. The company expanded its partnerships with AWS, Microsoft Azure, Google Cloud, and others, including a new pay-as-you-go offering on AWS Marketplace and a strategic collaboration agreement with AWS. Cockroach Labs also partnered with PwC UK to help financial institutions modernize their data infrastructure. With unprecedented tech outages disrupting people and businesses throughout 2024, the company remained focused on enhancing operational resilience. The State of Resilience Report polled over 1,000 senior technology leaders on the resilience strategies, priorities, and vulnerabilities facing enterprises today, highlighting key findings such as 86% of executives believing every minute of unplanned downtime risks losing customers.

AWS has announced Aurora DSQL, a distributed SQL database that is now in Preview. This new offering is designed to provide high scalability and strong consistency, with features such as ability to scale up and down, serverless, active-active, multi-region, and PostgreSQL compatibility. In contrast, CockroachDB offers a more mature product with better support for ACID transactions, automatic distribution of data across nodes, and strong consistency without the need for manual sharding or schema planning. CockroachDB also provides granular data placement capabilities, allowing users to specify where their data is stored geographically within specific regions to comply with data sovereignty laws. Additionally, CockroachDB offers better PostgreSQL compatibility, support for critical features such as UDFs and stored procedures via PL/pgSQL, foreign keys, triggers, views, sequences, and more. The product also provides regional resilience and seamless migrations, allowing users to deploy databases across multiple cloud providers or self-hosted environments with ease. Furthermore, CockroachDB offers a multi-cloud, hybrid deployment strategy, enabling businesses to leverage the best features and pricing from different providers while maintaining control over their infrastructure.

This guide provides best practices for operating CockroachDB on Microsoft Azure, covering backups, compute and storage considerations, security, data locality, performance testing, and survivability goals. It emphasizes the importance of implementing an effective backup strategy, leveraging Azure's robust services to safeguard distributed data, and tailoring a strategy that meets organizational needs. The guide also discusses best practices for compute type, storage type, and network configurations, as well as security measures such as authentication and authorization, and compliance considerations. Additionally, it covers performance testing and survivability goals, including high availability configuration, replication constraints, and network configuration. By following these best practices, users can optimize their CockroachDB deployment on Azure for enterprise-scale workloads.

The Digital Operational Resilience Act (DORA) is a landmark regulation in the European Union aimed at addressing the unprecedented risks associated with digital financial services, which have become increasingly dependent on modern technologies. DORA imposes requirements on financial institutions to enhance operational resilience and security, including ICT risk management strategies, incident management, digital operational resilience testing, third-party risk management, governance & oversight, information sharing, and vendor evaluation. To comply with DORA, technology leaders must take a proactive role in evaluating vendors and assessing their resilience capabilities, track record, and security maturity. This requires considering four primary dimensions: critical dependency, resilience capabilities, track record, and security & compliance maturity. By adopting the principles of DORA, technologists can build a stronger foundation for their organization and customers, position themselves as key drivers of innovation, security, and trust, and ultimately achieve wins in smoother onboarding processes, uninterrupted services, and avoiding fines from regulators.

CockroachDB is a distributed SQL database that offers a highly compatible PostgreSQL interface, allowing customers to easily migrate their applications to the new database without sacrificing performance or compatibility. The company has re-written the SQL layer from scratch to deliver the best performance and resilience possible, while also providing a familiar and easy-to-use SQL interface. In contrast to other vendors who have forked PostgreSQL, CockroachDB owns its own stack, resulting in less technical debt and the ability to optimize for customer experience. This enables the company to provide better distributed-SQL performance, simplify database migrations, and deliver improved resilience with features such as WAL failover and a cost-based optimizer. Ultimately, compatibility matters, but it's subjective and being 100% compatible is impossible; customers should look for databases that offer the features they need while still providing the resilience their applications demand.

The text discusses the implementation of a dual-approach monitoring system called Network Inspector in CockroachDB Cloud, which improves network observability and enables proactive detection and resolution of network issues before they affect customers. The solution combines two complementary approaches: proactive network probes that monitor DNS health and pod-to-pod connectivity, and network observation via Hubble, which enriches network metrics with contextual labels. By providing real-time visibility into network health, Network Inspector helps detect issues earlier and provides the detailed metrics needed for quick problem resolution.

We are excited to announce the launch of a new pay-as-you-go offering for CockroachDB Cloud on Google Cloud Marketplace, making it easier for teams to procure and manage costs through their existing Google Cloud billing account. This new offering provides enhanced flexibility, simplified subscription experience, streamlined billing, and aligns with our shared vision with Google Cloud, eliminating complexity in database management, enabling teams to focus on developing applications. With three tiers - Basic, Standard, and Advanced - now available through Google Cloud Marketplace, customers can easily deploy CockroachDB Cloud with predictable pricing and enterprise-grade capabilities, making it ideal for organizations seeking flexible cloud spending and operational savings.

The importance of operational resilience is highlighted by a recent survey from Cockroach Labs, which found that 55% of companies experience IT disruptions and outages on a weekly basis, resulting in costs ranging from $10,000 to over $1 million per incident. The survey also revealed that almost half of companies take two or more hours to fix outages, while nearly half acknowledge their organizations are falling short in resilience enhancement. Cockroach Labs' CEO Spencer Kimball emphasizes the need for proactive IT resilience strategies and highlights the growing momentum behind digital transformation as key drivers of this shift. Additionally, the survey's findings underscore the critical role of robust database solutions like CockroachDB in minimizing downtime and ensuring business continuity.

Legacy mainframe systems are increasingly seen as costly and inefficient for meeting modern digital demands. Migrating these systems to modern platforms offers enhanced scalability, flexibility, and cost savings. However, migration comes with complexities such as preserving critical business logic and data integrity. Organizations must meticulously plan, execute, and validate the migration process to minimize disruptions and ensure smooth operation. The essential steps in preparing for a successful legacy mainframe database and application migration include assessment and analysis of existing databases, identifying modernization targets based on business criticality, technical complexity, and alignment with strategic goals, defining measurable goals, balancing quick wins and long-term value, engaging stakeholders, and making a strong business case. Assessment and analysis involve evaluating the current setup, analyzing data structures and schema alignment, identifying dependencies, and assessing integration points. Identifying modernization targets requires prioritizing systems based on their impact on revenue, customer engagement, operational dependence, technical complexity, and alignment with strategic goals. Defining measurable goals involves setting short-term and long-term objectives, balancing quick wins and long-term value, and aligning the modernization strategy with organizational capabilities. Stakeholder engagement is crucial for gaining support and alignment from key organizational stakeholders. This can be achieved by communicating the strategic value of modernization, engaging stakeholders early and often, demonstrating quick wins and early successes, and highlighting employee impact and upskilling opportunities. Creating a strong business case involves presenting a detailed analysis of costs involved in maintaining legacy systems compared to the investment required for modernization, addressing potential risks associated with migration, examining use cases, assessing key performance metrics, ensuring data is ready for migration, and showing projected ROI over time.

CockroachDB release 24.3 introduces write-ahead log (WAL) failover to minimize the impact of transient disk stalls commonly seen in cloud storage. WAL failover uses a secondary disk as a fail-to for the write-ahead log, mitigating latency introduced by disk stalls. This feature is highly recommended for deployments on AWS, Azure, or GCP that use network storage devices or local SSDs and for on-premise installations where disk stalls are occurring. WAL failover helps CockroachDB maintain its reputation as a resilient multi-cloud distributed SQL database in the market.

City Storage Systems (CSS), a startup known for its brands Otter, CloudKitchens, and Lab37, was highlighted at the final installment of RoachFest 2024 in New York City. CSS aims to revolutionize restaurant infrastructure by building software to support growing food businesses and kitchens. Their goal is to deliver a database-as-a-service (DBaaS) for developers without worrying about platform reliability. At the event, Software Engineer at CSS, Frederik Stenum Mogensen, presented on "Low Latency Multi-Regional Microservices: Coordinating Data Locality." He discussed how his team developed an innovative pattern to ensure low latency in a multi-regional microservices environment using CockroachDB zone configurations and a Kubernetes operator. CSS's approach involves using Kubernetes operators for resource provisioning and data placement across tables, custom developer-defined resources for specifying desired states for data sharding and placement, defined business domains, mapped partitions to specific regions, and smart load balancing to route requests efficiently within regions. This solution maintains low latency and reliability in a multi-regional microservices setting by leveraging CockroachDB and Kubernetes while minimizing cross-region network data transfer, reducing costs, and improving performance.

Cockroach Labs has been named a finalist for the AWS Industry Partner of the Year - Travel and Hospitality - NAMER, highlighting their commitment to providing innovative solutions and services that drive modernization and efficiency. Their distributed SQL database, CockroachDB, ensures high availability, scalability, and resilience for hospitality businesses managing vast amounts of data across multiple locations. The company's work with a leading online travel agency demonstrates the impact of their technology in streamlining operations and enhancing customer experience. Cockroach Labs will be present at AWS re:Invent to share more about their solutions and partnership with AWS.

The text discusses RoachFest, a global event series for data professionals focused on operational resilience and performance at scale. Matt Stephenson, Senior Principal Software Engineer at Starburst, was a guest speaker at the final gathering in New York City. Starburst is the developer of the fastest and most scalable open hybrid data lakehouse. The company built Starburst Galaxy, a cloud-managed service for their distributed data lakehouse platform using Trino (formerly Presto), an open-source SQL query engine. Starburst chose CockroachDB to power its multi-region cloud lakehouse platform due to its high survivability and performance goals. The company has found that CockroachDB is always available even when entire regions fail, supports multi-region and multi-cloud deployments, and enables seamless service interruption management. Today, Starburst operates across 25 cloud regions on AWS, GCP, and Azure, with six regions using CockroachDB to guarantee roundtrip times of 40-80ms for queries coming from their services plane.