Computing Accurate Percentiles with DDSketch

DDSketch is a new sketch algorithm designed to accurately compute percentiles on large-scale monitoring data. It was developed by Datadog, which handles vast amounts of distributed data daily. Unlike existing state-of-the-art quantile sketch algorithms, DDSketch provides relative-error guarantees that better reflect users' needs when looking at latency plots. This makes it more memory-efficient and accurate than other sketches with rank-error guarantees. DDSketch has a small memory footprint and is highly performant, making it suitable for use in monitoring systems. It is currently being used at scale at Datadog and has open source implementations available in Java, Go, and Python.