September 2022 Summaries

Cockroach Labs has been named the top large company on Crain’s 2022 Best Places to Work in New York City list, marking the third consecutive year it has received this distinction. Ranked fifth overall among 100 NYC companies, Cockroach Labs emphasizes a workplace culture centered on resilience, innovation, and a balance between personal and professional commitments. In response to recent disruptions to traditional office operations, the company developed a scalable, distributed workplace model to foster flexibility, collaboration, and productivity. This approach has driven significant growth, adding over 125 employees in the past year, while maintaining a focus on inclusivity and diversity. Employees, referred to as "Roachers," contribute to a culture where continuous learning and support are prioritized, with unique initiatives like "Flex Fridays" and "Roach & Relaxation" days promoting work-life balance. The company is proud of its human-centered culture and invites interested individuals to explore career opportunities as it continues to expand.

Database outages continue to occur due to various factors, despite advancements in high availability systems designed to prevent such events. High availability has evolved from focusing on redundant hardware components to leveraging distributed systems that maintain service continuity even when individual nodes fail. However, outages still happen at major tech companies due to human errors, such as network configuration mistakes, which remain challenging to eliminate. Older enterprises often rely on outdated, non-cloud-native technologies that lack the resilience of modern systems, contributing to their vulnerability. To enhance data resilience, companies are encouraged to automate disaster testing, adopt self-healing databases, and implement data replication for redundancy. While cloud-native solutions offer a path to greater resilience, many organizations need motivation to transition away from legacy technologies.

RoachFest 2022, the first-ever CockroachDB customer conference, was held in New York City, bringing together data innovators and engineering experts to discuss and explore the database's applications and best practices. The event featured notable speakers from companies like Netflix, Twitter, DoorDash, and Hard Rock Digital, who shared insights on utilizing CockroachDB's distributed system for scalability and operational simplicity. Key themes included the shift towards everything-as-a-service to reduce operational burdens and the importance of understanding the nuances of distributed databases. Attendees engaged in discussions on topics like migrating from AWS Aurora, optimizing performance, and handling schema changes. The conference emphasized CockroachDB's compatibility with Postgres, providing developers with a familiar environment while leveraging the benefits of a distributed system. The event also highlighted the value of CockroachDB support in aiding users to make informed decisions and the vibrant community resources available, such as Cockroach University and forums. The conference concluded with networking activities hosted by partners like Google Cloud, AWS, and Red Hat, and promised future access to the recorded sessions.

Serverless computing has emerged as an essential paradigm in the evolution of cloud technology, offering significant advantages by abstracting complex operational tasks such as server provisioning and capacity management, and allowing developers to focus on building scalable and reliable systems. This approach democratizes cloud-native architecture, making it accessible to small and medium enterprises, and fostering innovation by freeing up technical teams to rapidly prototype and trial new products. Serverless databases further enhance this potential by eliminating the need for self-hosted legacy solutions and providing globally scalable, resilient, and cost-effective infrastructure. As an enabler of agile and iterative development, serverless computing is transforming industries like gaming, where startups can experiment and scale without incurring substantial upfront infrastructure costs. This shift is driving a new wave of developer-entrepreneurs who can test and release ideas with minimal barriers, highlighting serverless as a catalyst for both immediate business value and long-term survival in an unpredictable technological landscape.

Cockroach Labs has announced the general availability of CockroachDB serverless, a scalable and highly available relational database designed to handle provisioning, capacity, availability, scale, and maintenance automatically. The database aims to bridge the gap between the demands of modern cloud applications and the limitations of traditional databases like PostgreSQL and MySQL. CockroachDB serverless offers advanced features such as change data capture, automated backups, and a 99.99% uptime SLA, alongside a consumption-based pricing model. Additionally, Cockroach Labs has introduced CockroachDB MOLT, a new toolset to facilitate migration from legacy databases, and announced integrations with popular platforms like Vercel and HashiCorp Terraform to enhance development and operational workflows. The initiative reflects an ongoing commitment to simplifying database management and empowering developers to build applications efficiently without the operational burdens associated with traditional databases.

The blog post written by Cassie McAllister discusses how Nightfall, a company focused on preventing data loss across cloud applications, has integrated CockroachDB into their tech stack to manage metadata storage and act as a system of record. Nightfall initially managed a variety of databases, but as their customer base grew and the demand for data protection increased, they sought a more efficient, scalable solution that CockroachDB provided. CockroachDB's ability to scale across multiple regions, offer high availability, and maintain data locality requirements made it a strong fit for Nightfall's needs. It supports Nightfall's API gateway for processing high volumes of requests and their interactive analytics dashboard by providing a centralized and reliable data storage system. The blog also highlights Nightfall's migration to Kubernetes for enhanced scalability and offers tips for leveraging CockroachDB effectively, emphasizing its SQL compatibility and distributed nature, which allows the team to handle complex queries and data distribution with ease.

CockroachDB has introduced a feature that allows users to export their database logs to cloud-native services like AWS Cloudwatch or GCP Cloud Logging, providing enhanced self-service observability without needing technical support. This capability addresses the challenges of identifying transaction bottlenecks and auditing user actions by enabling direct access to logs, which is crucial for maintaining service level agreements and information security. The log export process involves configuring a cloud-native log service, setting up necessary IAM roles, and streaming logs in real-time, including SQL audit logs marked as redactable to protect sensitive data. Customers can also redirect these logs to third-party observability platforms if preferred. The integration with AWS and GCP requires specific configurations, such as cross-account IAM roles and trust relationships, ensuring secure log access and management. This feature is particularly beneficial for application teams needing to evaluate service providers based on manageability and security, as it provides seamless user onboarding, self-service capabilities, and robust data protection measures.

The blog post by Aydrian Howard describes the development of a serverless Ugly Sweater Voting Application designed to entertain a distributed team. The application features a serverless backend that facilitates SMS-based voting and real-time updates using technologies like Netlify, Twilio, CockroachDB, Next.js, and Pusher Channels. The app allows users to enter a contest by texting an image and name, and vote by texting the name of their choice, with Twilio managing the SMS functionality. CockroachDB serverless is utilized for scalable data storage, while Next.js is used for the frontend dashboard to display contest entries and votes. Pusher Channels enable real-time interactivity by updating vote tallies globally without needing manual page refreshes. Howard emphasizes the ease and scalability of the serverless architecture and encourages building with Pusher Channels based on their positive experience.

The shift toward cloud databases for operational data, historically lagging due to concerns over data sensitivity and mission-critical nature, is gaining momentum as the high costs and hiring challenges of on-prem systems become increasingly prohibitive. Managed cloud solutions are now more appealing due to their cost-effectiveness, scalability, and the expertise offered by providers, which often surpass the capabilities of in-house teams. In 2022, the demand for specialized roles like SREs and DBAs has surged due to increased venture capital investment, making it difficult and costly for companies to hire these experts. Meanwhile, the 2022 Cloud Report highlights that while the costs of managed cloud databases are significantly influenced by storage and data transfer expenses, most companies can opt for general-purpose storage without performance loss. This makes managed cloud databases an attractive option for operational data, offering flexibility and reducing the complexities of scaling global operations.

A serverless database is an innovative approach that embodies the principles of serverless computing, offering significant benefits such as reduced operational management, elastic scaling, built-in resilience, and consumption-based billing. This model allows developers to focus on coding rather than server maintenance, as server management tasks are abstracted and automated. Serverless databases are designed to be always available and fault-tolerant, providing instant access even during backend failures. They scale automatically based on workload demands, ensuring cost efficiency by charging only for the resources consumed. Additionally, serverless databases must meet specific requirements, including surviving failure domains, supporting geographic scale, offering transactional guarantees, and maintaining the elegance of relational SQL. This paradigm shift aligns with distributed systems principles and is gaining popularity as it simplifies operations and reduces costs, ultimately transforming databases into a seamless service akin to a SQL API in the cloud.

The blog post by Aydrian Howard provides an insightful look into the creation of a serverless gambling app designed for the Super Bowl, known as "Roacher Bowl Squares." The app aims to digitize the traditional "Super Bowl Squares" game, where participants wager on a grid-based system without requiring sports knowledge. Howard reflects on the technology stack choices, such as using Remix over Next.js, and emphasizes the utility of CockroachDB serverless for its elastic scaling capabilities. Other technologies like Prisma for database interactions, Netlify for deployment, and Pusher Channels for real-time updates are also discussed. The app offers a unique way for distributed users to engage in friendly betting during the "Big Game," and Howard shares the challenges encountered and decisions made throughout the development process.

Rebecca Taft's journey to Cockroach Labs began after completing her PhD at MIT, where she discovered her passion for databases under the guidance of renowned advisor Mike Stonebraker. Her decision to join Cockroach Labs, despite initial reservations about the startup's viability and the lack of female engineers, was driven by her admiration for the company's commitment to values like respect and transparency, as well as its open-source CockroachDB technology. Over the past five years, Taft has advanced her career from an individual contributor to the Engineering Manager of the SQL Queries team, witnessing the company's growth from a small startup to a successful enterprise with a $5 billion valuation and over 400 employees. She values the supportive work culture and has continued to engage with the academic community, contributing to database conferences and research, while also navigating challenges and opportunities within her team to enhance the stability and performance of CockroachDB.

Will Cross's blog post from September 8, 2022, delves into SQL performance best practices, emphasizing the pivotal role of indexes for optimizing database queries. Highlighting that efficient indexing can notably enhance performance, the article outlines several rules: filtering queries to avoid full table scans, sorting queries to expedite results, and "covering" queries to bypass table access entirely. Using CockroachDB as an example, the author demonstrates how using statement plans with EXPLAIN ANALYZE can provide insight into database behavior, facilitating informed indexing decisions. The post illustrates how constructing a composite index, such as one on both email and datetime, can reduce the workload by limiting row scans and sorts, thus significantly improving query execution time. A covered query, which avoids accessing the table by utilizing indexed data, is presented as the optimal solution for read performance. However, the author cautions against over-indexing due to the overhead of maintaining consistency with the table data, noting that large fields should be excluded from indexes to prevent excessive index size. The blog encourages leveraging tools like EXPLAIN plans and seeking community support for unexpected issues, while also promoting additional learning resources on SQL performance and best practices.

Tribal Credit aims to fill the gap left by traditional financial services for startups and SMBs by providing a scalable, globally distributed platform for agile online payments. Initially built on SQL Server and MySQL, the company faced challenges in scaling without increasing manual labor or tech debt, especially as demand grew during the pandemic. To meet these needs, Tribal Credit adopted CockroachDB for its ability to automate scaling, replicate data across regions, and maintain SQL compatibility, which facilitated their transition to a microservices architecture. This change allowed them to efficiently expand into new markets, including Latin America, while maintaining high availability and data consistency crucial for handling financial transactions. As they plan to enter additional regions like Chile, Colombia, Peru, and Brazil, the company continues to leverage CockroachDB's distributed capabilities to support their ambitious growth strategy and ensure service reliability.

Guy Harrison, CEO of alwaysNFT.cloud, discusses how the startup leveraged CockroachDB to develop a Web3 application aimed at ensuring the permanence of digital assets associated with NFTs. Despite the speculative nature of NFTs, they possess significant economic value and hold potential future utility as ownership certificates for various assets. The challenge lies in managing off-chain storage, with IPFS being a partial solution, as it helps maintain data integrity but requires NFT files to be actively "pinned." CockroachDB, with its distributed architecture, provides the scalability, transactional consistency, and real-time analytical capabilities needed to manage the large and dynamic NFT market. For startups, the serverless option of CockroachDB offers an attractive pay-per-use model that minimizes administrative overhead, making it a practical choice for small teams. The integration of CockroachDB with blockchain technology highlights its potential role in the evolving landscape of Web3, offering a robust solution for scalable, structured data storage.

Multi-region application architecture enhances system resilience and user experience by providing fault tolerance against regional failures, reducing latency for a global user base, and facilitating user growth in new regions. It ensures that applications remain operational even during disasters by distributing data across multiple regions, which also helps in maintaining low latency by keeping data close to users. This architectural approach supports scaling flexibility, allowing organizations to add new regions easily and optimize application performance. Additionally, it helps businesses comply with various data privacy laws by ensuring data is stored within specific geographic boundaries. The architecture's benefits make it an attractive option for companies aiming to expand their reach and improve service reliability.