Company
Date Published
Author
John Meehan
Word count
1599
Language
English
Hacker News points
None

Summary

YugabyteDB uses RocksDB as its internal persistent data storage, which utilizes DocDB's highly customized document store. RocksDB's data compaction algorithms are used for performance and space optimization. Compactions in YugabyteDB aim to minimize read latency by reducing the number of files accessed to find a key-value pair, and also serve as a form of garbage collection by purging unnecessary data from disk during compaction. The type of background compaction algorithm used is size amplification or read amplification, which trigger when certain thresholds are reached regarding file sizes and read amplifications. In YugabyteDB, TServer flags can be used to tune compaction resources, including the number of threads assigned to a thread pool for compactions and flushes, as well as limits on SST files allowed per tablet to help prevent behind-on-compaction issues. Full compactions are scheduled to improve cluster performance by minimizing space amplification and improving read performance over time.