Compression in ScyllaDB, Part Two

In the second part of the blog series on compression in ScyllaDB, Kamil Braun delves into practical testing of various compression algorithms to evaluate their performance in terms of compression ratios, speeds, and reactor stalls. Using a structured and repetitive data set from IMDB, Braun benchmarks algorithms like DEFLATE, LZ4, Snappy, and ZStandard, analyzing their effectiveness across different chunk lengths and compression settings. The tests reveal that DEFLATE and higher levels of ZStandard achieve the best compression ratios, while LZ4 and Snappy excel in compression and decompression speeds, sometimes outperforming no compression due to reduced disk write volumes. The study also explores the impact of these algorithms on ScyllaDB's reactor stalls, finding minimal issues with fast algorithms, but noting potential stalls with DEFLATE and higher ZStandard levels at larger chunk sizes. Braun concludes by advocating for compression use, noting that it often enhances performance and space efficiency, and suggests tailoring settings to specific workloads through dedicated testing.