Maximizing Disk Utilization with Incremental Compaction

The blog post discusses the limitations of the Size-Tiered Compaction Strategy (STCS) used in databases like ScyllaDB, which requires maintaining at least 50% of disk space free to ensure smooth operations during data compaction, leading to significant storage inefficiency and increased costs. To address this, the Incremental Compaction Strategy (ICS) has been introduced, which combines elements of both Size-Tiered and Leveled compaction strategies to drastically reduce temporary space overhead during compaction. ICS achieves this by breaking down large SSTables into smaller fragments, allowing for more efficient space usage without increasing read or write amplification. The strategy is available in ScyllaDB Enterprise releases from version 2019.1.4 onwards and allows for greater disk utilization, enabling users to use up to 90% of available disk space without the risk of running out of space during compactions. The post recommends ScyllaDB Enterprise users transition from STCS to ICS to maximize disk usage and minimize costs, while maintaining performance efficiency.