More Efficient Range Scan Paging with ScyllaDB 3.0

ScyllaDB Open Source 3.0 introduces stateful range scans, building on the stateful paging mechanisms introduced in version 2.2 for singular partition queries, significantly improving efficiency in handling range scans, which are queries that don't use the partition key in the WHERE clause. This new feature enhances throughput by 30% and reduces disk reads by 39% and disk operations by 73% compared to its predecessor, version 2.3. The stateful range scans utilize a special cache to save the reading pipeline, allowing queries to resume where they left off, which reduces the computational load on the cluster and shifts the performance bottleneck from the disk to the CPU. This change allows ScyllaDB to handle a greater number of reads per second, accommodating more concurrent loaders and improving overall database performance. These optimizations make range scans more efficient and reliable, providing significant improvements over the previous stateless implementation, and are expected to offer even more benefits with non-fully compacted tables.