How we could save petabytes of cache storage with Zstandard and Pingora

Cloudflare prototyped Cache Transcoding, a Pingora-based system that uses Zstandard compression to store eligible cache objects in compressed form, aiming to offset rising memory and disk costs while reducing inter-data-center bandwidth use. The system compresses unencoded, compressible text responses such as HTML, JSON, CSS, and JavaScript of at least 4 KiB when they enter the cache, retains the compressed representation on disk and across Tiered Cache transfers, and decodes it only before client delivery. Initial controlled tests found that eligible assets shrank by about 2.8 times on average, offering substantial effective cache-capacity gains in exchange for a modest CPU cost, with encoding paid once per cache fill and decoding required on each response. Already compressed media, precompressed content, binary data, range requests, and unknown-length responses are excluded because they offer limited savings or add complexity. Testing across more than one million requests and 10 cache servers validated cache-hit, cache-miss, and tiered-transfer behavior, though Cloudflare notes that broader real-world testing is needed before applying the measured compression ratio across its network.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.