Home / Companies / QuestDB / Blog / Post Details
Content Deep Dive

Importing 300k rows/sec with io_uring

Blog post from QuestDB

Post Details
Company
Date Published
Author
Andrey Pechkurov
Word Count
2,486
Language
English
Hacker News Points
-
Summary

QuestDB's latest update, version 6.5, introduces a highly optimized CSV file import feature using the SQL COPY statement, significantly enhancing the speed and efficiency of data ingestion. Leveraging io_uring, a modern asynchronous I/O interface, QuestDB has improved its import performance, surpassing several well-known OLAP and time-series databases in the ClickBench benchmark, with only ClickHouse achieving faster results. The new import method, which processes data in parallel through multiple phases, has reduced import times dramatically, particularly when using high-memory setups or NVMe SSDs. This update allows for in-order data ingestion and reduces the need for complex partition re-writes, optimizing data layout on disk for immediate querying. The improvements in QuestDB's import capabilities were driven by insights gained from the ClickBench benchmark, which also highlighted opportunities for enhancing query stability and performance. The update positions QuestDB closer to a practical alternative to ClickHouse for time-series data analysis and sets the stage for further innovations, including potential applications of io_uring in QuestDB Cloud.