Company
Date Published
Author
Jason Moiron
Word count
340
Language
English
Hacker News points
None

Summary

Jason Moiron from Datadog is releasing two new cgo bindings for compression libraries czlib and zstd, which have been used in production by Datadog. The goal was to improve performance of their data pipeline using zlib compressed messages. czlib is a modified version of the vitess project's cgzip package that uses zlib wrapping instead of gzip headers, resulting in faster encoding and decoding speeds. In contrast, zstd is a relatively new fast compression library from Yann Collet, which compresses slightly faster than zlib at level 6 and decompresses much faster, making it a great general-purpose zlib replacement. The zstd Go binding exposes common interfaces like stream compression, compression levels, and pre-computed dictionaries, making it functionally similar to the zlib interface with some additional features.