Ingesting live market data from Databento into QuestDB

QuestDB Enterprise 4.0.0 introduces a Databento connector that streams live or historical market data into QuestDB through a configuration file, replacing the Python-based ingestion approach required by the open-source edition. Using Databento’s EQUS.MINI US equities feed, the connector writes top-of-book updates and trades into automatically created, day-partitioned tables with nanosecond timestamps, order-book arrays, WAL support, and SQL-queryable schemas. The post demonstrates QuestDB SQL features including LATEST ON for current quotes, SAMPLE BY for OHLCV bars and activity analysis, and ASOF JOIN for relating trades to prevailing quotes, while Web Console notebooks provide live dashboards without requiring Grafana. Tests from a pre-market session show that retaining each update can expose spread changes, timestamp latency distributions, and millisecond-scale bursts of quote cancellations. The connector uses acknowledgement-based checkpoints, Databento intraday replay, and count-based replay deduplication to recover from restarts without gaps or accidentally discarding genuine identical records. During temporary QuestDB outages, it buffers incoming data and preserves original event timing when writes resume; replication configurations can also support failover to a promoted primary. For long-term retention, daily partitions can be transferred as Parquet files to object storage while remaining accessible through SQL, supporting both operational analysis and archival market-data workloads.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.