Time-series and analytical databases walk into a bar
Blog post from QuestDB
QuestDB is an open-source time-series database optimized for handling demanding workloads with ultra-low latency and high ingestion throughput, making it suitable for applications like trading floors and mission control. Written primarily in Java, with elements of C++ and Rust, it features a multi-tier storage engine and supports SQL with specialized extensions for time-series data, ensuring data portability without vendor lock-in. The database utilizes techniques from high-frequency trading systems, such as zero garbage collection on the hot path and an in-house API, to enhance performance. QuestDB's query engine is designed for parallel, vectorized processing and includes a JIT compiler, enabling efficient real-time data ingestion and querying. Although primarily a time-series database, QuestDB has been enhanced to support analytical queries more effectively by implementing optimizations such as specialized hash tables for parallel aggregation and expanding SQL functions. These improvements followed a two-year effort involving community contributions and benchmarking against standards like ClickBench, resulting in a significant performance boost. QuestDB continues to evolve with a focus on enhancing analytical capabilities, particularly for high cardinality queries.