QuestDB's QWP under the hood: Gorilla timestamp compression

QuestDB’s QWP protocol reduces time-series timestamp transfer sizes by using Gorilla encoding, a technique derived from Facebook’s Gorilla database that stores changes in timestamp intervals rather than full timestamp values. After transmitting the first two timestamps in full, QWP encodes the delta of delta between subsequent values, allowing regularly spaced timestamps to cost as little as one bit each and using variable-size bit buckets for larger timing changes. Tests described with live crypto and FX trade data show compression ranging from about 1.6x for volatile nanosecond-resolution data to roughly 57x for long, regular microsecond series, while sampled data with occasional missing intervals still achieves substantial savings. Gorilla encoding is advantageous whenever every delta-of-delta fits within a signed 32-bit range, but QWP falls back to plain 64-bit timestamps for an entire column batch if a value exceeds that limit, such as after long pauses in nanosecond data. The post also distinguishes Gorilla’s timestamp-focused implementation from Facebook’s original value compression scheme and previews QWP’s use of varints, ZigZag encoding, and bit packing for other data types.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.