Aeron and QuestDB: building open infrastructure for capital markets data

QuestDB is an open-source time-series database designed for demanding workloads, such as those found in trading floors and mission control, offering ultra-low latency, high ingestion throughput, and a multi-tier storage engine. It integrates with Aeron, a high-performance stack for low-latency applications, to support real-time market data and time-series analytics, crucial for pre-trade, post-trade, and surveillance tasks in capital markets. This integration allows firms to handle both the immediate processing of millions of messages with microsecond latency and the long-term storage of queryable historical data, supporting regulatory compliance and advanced analytics. The architectural design separates the "hot path" for immediate data processing from the "history path" for long-term data storage and querying, facilitating exactly-once semantics and enabling detailed analysis, backtesting, and reproducibility of trading policies. The use of open-source infrastructure, such as QuestDB and Aeron, aligns with regulatory demands and supports AI and machine learning applications by providing open, machine-readable formats and ensuring data portability with no vendor lock-in. This approach is increasingly adopted in capital markets as firms seek to rebuild their tech stacks on open foundations, enhancing both the speed of operations and the analytical capabilities while maintaining high availability and resilience.