How ScyllaDB Simulates Real-World Production Workloads with the Rust-Based “latte” Benchmarking Tool

ScyllaDB employs a customized version of the Rust-based benchmarking tool, "latte," to simulate real-world production workloads that are specific to its high-performance distributed database needs. Originally designed for latency measurements in healthy database clusters, ScyllaDB's fork of latte includes unique features such as preferred datacenter and rack support, sine-wave rate limiting for workload prioritization, and real-time HDR histogram data streaming to avoid RAM leaks. The tool's flexibility allows the ScyllaDB testing team to define any schema and workload functions, using rune scripts as business logic. This enhanced version aims to address specific challenges like coordinated omission in latency measurements and saturated database impacts, enabling the testing of complex customer scenarios with controlled disruptions. By adopting latte, ScyllaDB can quickly reproduce customer workloads in test environments, which is crucial for troubleshooting issues like latency spikes and throughput drops.