September 2026 Summaries

QuestDB’s guide explains why equality-based SQL joins are often unsuitable for independently timestamped time-series data and presents specialized joins designed around temporal relationships. ASOF JOIN attaches the most recent matching state to each event and can use TOLERANCE to prevent stale matches, WINDOW JOIN aggregates all observations within a time range around an event, and HORIZON JOIN retrieves the latest values at multiple fixed offsets before or after events for analyses such as markout curves. LT JOIN differs from ASOF by requiring a strictly earlier match, making it useful for bucketed data and avoiding look-ahead bias, while SPLICE JOIN preserves every update from two equally important streams by pairing each with the prevailing value from the other. The guide also covers LATERAL JOIN for executing queries per outer-row grouping, such as finding top trades per symbol, and describes restrictions on combining join types, recommending common table expressions when further aggregation or processing is needed. Using simulated foreign-exchange trades and quotes, it emphasizes selecting a join based on the desired output shape, from one contextual value per event to summaries, offset-based observations, previous bars, or a unified record of changes across two feeds.

QuestDB’s cold storage uses plain Hive-partitioned Parquet files, allowing the same data to be registered without copying into multiple table formats, including Iceberg and DuckLake, while remaining directly queryable through Parquet readers. Both formats provide metadata for file membership, schemas, snapshots, and statistics, but Iceberg stores metadata files alongside data in object storage and supports broad lakehouse interoperability, whereas DuckLake 1.0 stores metadata as rows in a SQL catalog such as SQLite, PostgreSQL, or DuckDB, enabling indexed metadata queries and avoiding manifest-file maintenance. DuckLake can register QuestDB files with `ducklake_add_data_files`, infer schemas from Parquet, and query the resulting table in DuckDB, although DuckDB may expose QuestDB nanosecond timestamps at microsecond precision. Catalog synchronization is required when QuestDB adds or removes cold partitions because neither format automatically discovers files; DuckLake registrations must be carefully diffed to avoid duplicate row counts, while removals require re-registering the surviving file set. Since DuckLake maintenance may otherwise rewrite or delete registered files, compaction should be disabled and storage access can be limited to read-only credentials. Iceberg is positioned for broad organizational and multi-engine lakehouse access, while DuckLake is suited to simpler, fast DuckDB-centric analysis, illustrating how open Parquet storage lets table formats remain a flexible per-use-case metadata choice rather than a permanent data commitment.

QuestDB uses Apache Parquet to store older time-series partitions cheaply in portable columnar files, while Apache Iceberg provides a separate metadata layer that turns collections of such files into versioned, transactional, interoperable tables. Parquet enables compression, column projection, and predicate pushdown but does not independently handle atomic multi-file commits, concurrent writes, snapshots, schema evolution, or updates, whereas Iceberg addresses these needs through catalogs, manifests, and immutable snapshots. QuestDB manages these capabilities for its own tables through internal metadata and write-ahead logging, but external engines such as Spark, Trino, Snowflake, and Databricks see raw cold-storage Parquet files rather than a shared table. Existing QuestDB Parquet partitions can be registered as Iceberg tables without copying or rewriting data, allowing QuestDB to remain the low-latency engine for hot and cross-tier queries while other tools access governed historical data. Until native Iceberg integration arrives, users must run a scheduled synchronization process to register newly added Parquet files and remove deleted ones, with particular attention to catalog and storage credentials as well as timestamp and UUID type compatibility.