ScyllaDB Open Source 3.1: Efficiently Maintaining Consistency with Row-Level Repair

ScyllaDB Open Source 3.1 introduces a significant improvement in its data synchronization process with the implementation of a row-level repair algorithm, which enhances performance by operating at a more granular level compared to the previous partition-level approach. This new method minimizes data transfer by exchanging only mismatched rows between nodes, rather than entire partitions, thereby reducing unnecessary data transmission and resource consumption. Additionally, the new repair process is more efficient, achieving faster synchronization times and reduced data transfer costs, particularly in scenarios where only a small percentage of data is out of sync. Benchmark results demonstrate considerable improvements, with the new algorithm being up to 6.78 times faster in some cases and using significantly less bandwidth. The enhancements also include increased parallelism and the use of a faster, non-cryptographic hash, resulting in reduced CPU, disk, and network resource usage and freeing up resources for online requests. These changes not only improve the efficiency and speed of repairs but also reduce the likelihood of failures during the repair process, with further improvements planned for future ScyllaDB versions.