Better “Goodput” Performance through C++ Exception Handling

Piotr Dulikowski discusses the unexpected performance issues encountered while implementing per-partition query rate limiting in ScyllaDB, specifically focusing on C++ exception handling. Despite initial attempts to manage overloads by rejecting excess queries to maintain stable goodput, the handling of exceptions, particularly in large volumes, became a bottleneck due to the CPU-intensive nature of rejected operations. This issue was compounded by the use of Seastar, a C++ framework, which, while adept at managing asynchronous tasks, struggled with exception handling during high-load scenarios. To address these issues, the team explored two approaches: using boost::result to return errors instead of throwing exceptions, and implementing a custom utility for inspecting std::exception_ptr without rethrowing, thereby reducing the overhead of exception handling. Benchmarks comparing ScyllaDB versions 5.0.3 and 5.1.0-rc1 demonstrated that these optimizations significantly improved throughput and goodput, allowing the updated version to handle higher write rates with fewer timeouts, thus enhancing overall performance under stress.

No tracked trend matches for this post yet.