More Efficient Query Paging with ScyllaDB 2.2

ScyllaDB 2.2 introduced significant improvements in query paging by transitioning from stateless to stateful queries, addressing performance issues with large partitions and increasing throughput by up to 2.5 times. Previously, stateless paging discarded all query-related data at the end of each page, leading to redundant work and lower efficiency. The new stateful approach retains the query state across pages, reducing initialization overhead and improving resource management, particularly for disk read scenarios. This is achieved by using a "querier" object stored in a cache, which ensures that the work done for one page can be reused for subsequent pages. While the new method enhances throughput for disk reads, it required optimization for network-constrained environments where the introduction of sticky replicas initially reduced performance. Ultimately, stateful paging has demonstrated notable efficiency improvements, particularly in scenarios involving large data partitions.