Company
Date Published
Author
Andy Kimball
Word count
2634
Language
English
Hacker News points
None

Summary

Cockroach Labs has introduced a new cost-based SQL optimizer in its 2.1 release to enhance the performance and scalability of CockroachDB, particularly in handling complex queries. Unlike heuristic optimizers, which rely on preset rules that can sometimes fail to find the most efficient query execution plan, the cost-based optimizer evaluates multiple execution plans by assigning a cost to each, selecting the one that minimizes resource usage and execution time. This approach allows for more accurate and flexible query optimization, leveraging detailed database statistics to make informed decisions. The development of this optimizer involved a focused team effort and educational initiatives to understand state-of-the-art optimization techniques. The new optimizer supports features such as correlated subqueries and the automatic planning of lookup joins, and it employs a data structure called a memo to efficiently manage the numerous potential execution plans for complex queries. This structure helps in storing and evaluating plans without excessive memory use by recording only unique plan components. Cockroach Labs plans to continue refining these optimizations and invites community contributions and potential recruits to join their efforts in advancing distributed SQL engine technology.