Heat Weighted Load Balancing

ScyllaDB 2.0 introduces a feature called heat-weighted load balancing to address performance issues that occur when a node in a ScyllaDB cluster loses its cache, such as during rolling upgrades. Traditionally, when a node restarts with an empty cache, the equal distribution of workload among nodes can lead to overloading cold nodes, resulting in higher latency and reduced throughput. Heat-weighted load balancing mitigates this by allocating fewer requests to the cold nodes, allowing their caches to gradually warm up without overwhelming them with disk I/O operations. This approach adjusts the distribution of requests based on the cache hit rates of each node, aiming to equalize cache miss rates across the cluster. The solution involves complex algorithms to manage these adjustments, especially when consistency levels require multiple replicas to handle requests. Experimental results demonstrate that this method successfully maintains low latency during node cache warm-ups, ensuring consistent performance.