Measuring real-time performance per dollar under continuous load: CostBench’s first end-to-end results

CostBench evaluates real-time analytics performance per dollar by measuring the continuous path from low-latency ingestion through data preparation and concurrent query serving, arguing that systems must maintain columnar layouts, pruning-friendly organization, and current pre-aggregations to keep fresh data efficient to query. Its first benchmark streamed 113.2 billion stock-market quote records at a target rate of one million rows per second into ClickHouse Cloud, Snowflake, BigQuery, and Redshift Serverless using vendor-recommended push-based ingestion paths, while scheduled aggregate and event-level drill-down queries ran throughout. The benchmark used a shared client and workload, aligned resources where possible, disabled result caching, and scored platforms by combining fresh-data-path cost, normalized query-serving cost, and total query runtime, while excluding database storage and several billing-related factors. According to the reported results, ClickHouse Cloud had the lowest ingestion-preparation cost, query-serving cost, and accumulated runtime, with the other tested platforms scoring 412 to 1,996 times worse on the end-to-end metric, compared with a previously reported 32 to 101 times worse gap in query-side-only testing. The authors state that forthcoming provider-specific analyses will examine how architectural and billing differences contributed to these outcomes, while future benchmark rounds may include pull-based ingestion and Databricks after its Lakehouse/RT offering becomes generally available.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.