Improvements to ScyllaDB Streaming Performance

ScyllaDB's streaming process, a vital internal operation for data movement between nodes, has undergone significant improvements in its Open Source 3.0 release, enhancing efficiency and performance. The new streaming mechanism eliminates the use of memtables by directly writing data to SSTables on disk, leading to reduced memory and CPU usage, as well as fewer compactions. This upgrade utilizes the Seastar RPC Streaming interface, which improves bandwidth by 240% and reduces the rebuild operation time by 70%, as demonstrated in tests with clusters on AWS. These enhancements are set to accelerate cluster operations such as adding, decommissioning, and repairing nodes, and are part of ScyllaDB's effort to optimize its distributed database system.