July 2022 Summaries

In the rapidly evolving landscape of cloud computing, traditional caching solutions are falling behind due to their reliance on instance-based provisioning, which requires extensive planning and often leads to inefficiencies and high costs. Momento introduces a serverless, resource-based caching service designed to align with modern cloud capabilities, offering dynamic scaling without pre-provisioning, pay-per-use pricing, and superior performance characterized by low latency and high throughput. By leveraging shared infrastructure, Momento provides a more resilient and adaptable caching solution that minimizes downtime and optimizes resource utilization, making it suitable for both development and production environments. This innovative approach aims to modernize caching by eliminating the complexities and limitations associated with legacy systems, allowing developers to focus on building applications without the constraints of traditional infrastructure.

Momento Cache, an open-core caching engine built on Twitter's Pelikan, has been optimized for Google's Arm-based Tau T2A VMs to enhance performance and developer productivity by focusing on consistent tail latencies. By rewriting Pelikan in Rust and implementing multi-worker support, the team achieved significant throughput improvements without modifying the code, primarily through strategic systems tuning like core pinning. This approach allowed them to reach over a million operations per second (OPS) on a single T2A-standard-16 VM while maintaining a 2ms service-level objective (SLO). The optimizations, which included isolating network threads to specific cores, not only tripled the throughput but also significantly reduced tail latencies, offering a high return on investment for those concerned with performance at high loads. These tuning techniques also benefited x86-based C2 VMs, though the improvements were slightly less dramatic, showcasing the potential of core pinning in optimizing cache performance across different architectures.

Building high-performance systems requires focusing on tail latencies rather than median or average latencies, as these can significantly affect user experience, particularly in applications making multiple backend calls. A robust test harness is crucial for rapidly evaluating performance optimizations, exemplified by the use of rpc-perf at Twitter and its integration with Momento’s grpc protocol and OpenTelemetry for enhanced visualization. Establishing Service-Level Objectives (SLOs) for each system component helps ensure that performance standards are met and improved upon over time, balancing innovation with quality. Continuous performance testing in the CI/CD pipeline is essential to prevent performance regressions from reaching production, with performance canaries aiding in the prompt detection of issues.