ClickHouse vs Prometheus for High Cardinality, Part 1: Understanding the Problem

High cardinality poses significant challenges in traditional time-series systems like Prometheus due to the creation of numerous unique time series, each carrying overhead in terms of memory, indexing, and lifecycle management. In observability workloads, this complexity arises from the many unique label combinations that define metrics, leading to increased memory consumption and potential system instability, particularly in environments with high churn or ephemeral workloads. Prometheus, while efficient for moderate numbers of long-lived series with regular intervals, struggles with high cardinality as it leads to memory pressure and necessitates compromises such as limiting labels or reducing metric resolution to manage cardinality explosion. This impacts both write-time performance, where each new series incurs overhead, and read-time performance, where broad queries across high-cardinality dimensions require extensive data processing. These constraints often force users to trade off between visibility and system stability, highlighting the need for alternative approaches, like ClickHouse, which handle such workloads differently through column-oriented storage and other optimizations.