Company
Date Published
Author
Alexander Kuzmenkov
Word count
1778
Language
English
Hacker News points
None

Summary

ClickHouse`, a database management system, faces challenges in testing due to its complex task of interpreting query languages on persistent states managed in a distributed fashion. Despite extensive automated testing, new bugs and regressions continue to appear. To address this issue, the authors developed an `AST-based Query Fuzzer`, which uses property-based testing techniques to generate random SQL queries that test the system's behavior under various conditions. The fuzzer leverages a large corpus of human-written SQL queries from regression tests, allowing it to efficiently cover a wide range of possible permutations. By incorporating the fuzzer into their continuous integration process, the authors were able to identify over 200 bugs, including serious logic errors and memory issues, and improve the overall reliability of the system. The approach also benefits from the use of assertions and runtime checks, such as those provided by compiler sanitizers, to ensure that errors are distinct from user-induced errors.