July 2020 Summaries

Cockroach Labs has announced updates that improve compatibility with SQLAlchemy, a popular ORM for Python projects. These updates include new support for SAVEPOINT and improved transaction retries. A tutorial on building a "Hello World" app using SQLAlchemy and CockroachDB is available for Python developers interested in getting started with an easy SQL database.

The article discusses the growing trend of deploying databases in the cloud, with Gartner predicting that by 2022, 75% of all databases will be deployed in this manner. It highlights the challenges faced when scaling traditional databases to meet the demands of microservices architecture and how distributed SQL databases like CockroachDB can help overcome these issues. The article also mentions a webinar featuring Jim Walker, VP Product Marketing at Cockroach Labs, and Chris Casano, Solutions Engineer at Cockroach Labs, discussing the capabilities of Distributed SQL and how it delivers features that traditional databases cannot.

The European Court of Justice has abolished the Privacy Shield agreement between the U.S. and EU, which allowed American companies to store data about Europeans outside of the region. This move is in line with the EU's commitment to protect its citizens' data privacy and extends its leadership in General Data Protection Regulation (GDPR). The end of Privacy Shield presents significant compliance challenges for organizations due to varying regional data regulations, but those that have adopted a cloud-native approach are better prepared. CockroachDB offers geo-partitioning capabilities that allow data to be tied to a location and automate the process of moving data domiciled in one location to another, ensuring adherence to privacy obligations without operational complexity or high costs.

CockroachDB's 20.1 release introduces lateral joins, which can significantly reduce the lines of code needed for business analytics. This feature allows data sources to reference columns provided by previous data sources in the same query. An example using a fictional ride-sharing company, MovR, demonstrates how lateral joins can be used to run analytic queries directly in CockroachDB. The article provides an overview of setting up CockroachDB 20.1 and presents a step-by-step guide on implementing lateral joins for calculating the average time between customers' first and second rides, which is crucial for increasing customer loyalty and profitability.

CockroachDB has implemented a cost-based query optimizer from scratch, which is continually being improved. In the latest release (v20.1), significant improvements have been made in various areas such as foreign key checks, limit "hints", recursive common table expressions and statement diagnostics bundle. The new implementation of foreign key checks allows for locality-sensitive index selection, while soft limits provide more accurate cost estimations by considering early execution completion. Additionally, the support for recursive CTEs has been added, making SQL a Turing-complete language. A statement diagnostics bundle has also been introduced to simplify the collection of information when reporting issues.

Cockroach Labs has implemented strategies to combat unconscious bias in their hiring process. They have removed resumes from interviews and replaced them with experience levels, thesis focus for candidates with PhDs, and current year in school for interns. This approach helps mitigate biases such as affinity bias, halo/horn bias, and similarity bias. The company prepares both the candidate and interviewer for this process to ensure a fair and effective evaluation of skills required for the job.

Archer Zhang, a backend engineering intern at Cockroach Labs, has improved the algorithm behind some SQL operators in CockroachDB's vectorized execution engine to reduce memory usage and speed up query execution. Specifically, he focused on improving the unordered distinct operator by optimizing its performance and reducing its memory footprint. The new algorithms address existing inefficiencies and can provide a significant boost in speed for certain types of queries. These improvements are part of CockroachDB's 20.1 release.

Many growth-stage companies are choosing CockroachDB over PostgreSQL due to its ability to scale easily and offer transactional consistency. While startups may initially be tempted by the familiarity of Postgres or MySQL, CockroachDB's compatibility with these databases allows for a smooth transition. Additionally, CockroachDB's compatibility with Kubernetes and other cloud-native services makes it an attractive option for companies looking to build scalable applications from the start.

Kubernetes is a popular container orchestration platform used for managing state and storage. However, it can have a steep learning curve, especially when dealing with distributed systems and data pipelines. One of the biggest challenges in Kubernetes has been maintaining state and consistency of that state. Persistent volumes and StatefulSets were introduced to address this issue, allowing pods to bind persistent volume claims and maintain state across pod restarts. CockroachDB on Kubernetes can simplify these challenges by providing a single executable with a common gateway to the database through every node, making it easier to operate in production environments.