Building a garbage-free network stack for Kafka streams

QuestDB is an open-source time-series database designed for high-performance and latency-sensitive environments, such as trading floors and mission control. It offers ultra-low latency, high ingestion throughput, and a multi-tier storage engine, while ensuring data portability and AI-readiness through native support for Parquet and SQL. To address the challenges of garbage collection, which can introduce unpredictable pauses and resource overhead, QuestDB has developed a garbage-free network stack that bypasses Java's native non-blocking IO, using a custom notification system to manage network connections efficiently. This implementation includes an IODispatcher that mimics IO notification systems like epoll and kqueue, and a worker thread model that processes incoming data with minimal resource allocation. Additionally, QuestDB's network stack now supports Kafka Connect, allowing reliable ingestion of time-series data from Kafka topics without the overhead of garbage collection. The open-source nature of QuestDB invites community feedback and collaboration on its unique approach to non-blocking and garbage-free IO.