Community contribution from Alex Pelagenko improving our HTTP server
Blog post from QuestDB
QuestDB is an open-source time-series database known for its ultra-low latency, high ingestion throughput, and flexibility, supporting demanding workloads such as those on trading floors and mission control. It features a custom HTTP server with a non-blocking IO stack that uses a thin layer of JNI OS abstraction, handling inbound and outbound traffic with state machines. The author contributed to QuestDB by addressing the issue of data availability interruptions, a challenge due to the single writer model where tables are locked during certain operations like CSV imports. By implementing a queuing system, the author enabled the handling of requests with data availability errors through a priority queue, which idle threads process and retry at exponentially increasing intervals. This enhancement ensures that users do not encounter errors or need to retry operations themselves. The author appreciated the support received from the QuestDB team in understanding the complex HTTP stack and resolving the multi-writing challenge.