October 2022 Summaries

Serverless applications, known for their scalability, speed, and cost-effectiveness, benefit significantly from effective caching strategies to enhance performance and reduce costs. While local caching in serverless environments like AWS Lambda can improve efficiency by reusing resources within an execution environment, it has limitations due to its ephemeral nature and scope. A centralized caching approach, such as using Momento, a truly serverless cache, allows data to be shared across execution environments and microservices, enabling faster data access and reducing redundant calls to downstream services. This serverless caching solution eliminates the management overhead associated with traditional caching systems like AWS DAX or ElastiCache, providing a scalable, low-cost alternative that helps overcome the data size limitations of AWS services. By implementing a centralized cache, developers can enhance the performance of serverless applications, facilitate service-to-service communication, and simplify the architecture without relying on workarounds to avoid service limits.

In a narrative that intertwines a spooky Halloween theme with a real-world engineering challenge, the author recounts the experience of launching a video streaming service for a major sports league, which initially faced significant issues with overprovisioning on Amazon DynamoDB and Redis. The service's unforeseen data demands led to substantial costs, as the team had to transition from DynamoDB to a managed Redis solution, resulting in an overabundance of underutilized resources due to the high instance costs and single-threaded nature of Redis. The story highlights the challenges of managing cloud costs and the inefficiencies of paying for idle resources, ultimately leading to the development of Momento's serverless caching product, which emphasizes a fair pricing model that charges only for the data actually used, promising a more cost-effective and scalable solution without the complexities of traditional caching services.

Ellery Addington-White explores the integration of Momento Cache with Google Cloud Platform (GCP) to significantly reduce latencies in Cloud Run and Firestore, achieving a 98% reduction for /users and /cached-users endpoints and a 99.7% reduction for /followers and /cached-followers endpoints. The experiment involved setting up APIs to serve a social network app, where Firestore served as the NoSQL database alternative to AWS DynamoDB. Addington-White utilized Cloud Run for runtime due to its simplicity and future portability, and leveraged Google Cloud Metrics for monitoring, finding the GCP developer experience efficient and rewarding. By adjusting settings to align with AWS Lambda configurations, including CPU, memory, timeout, and container concurrency, he was able to achieve consistent sub-10ms response times and noted potential cost savings and enhanced scalability. The author encourages engineers using Firestore or GCP for unpredictable workloads to consider Momento for improved performance and cost-efficiency.

The blog post explores the performance tuning of Momento's Python cache client, aiming to optimize its efficiency and user experience in a serverless cache setup. It delves into the specifics of configuring gRPC channels, highlighting that unlike the JavaScript client, increasing the number of channels in Python doesn't improve performance due to differences in gRPC library implementations. The post emphasizes the role of CPU as a bottleneck in single-process Python runtimes and explores how switching to the uvloop engine can enhance performance by reducing CPU usage, resulting in a 10% improvement. Testing on an AWS EC2 instance showed that with 50 concurrent requests, the Python client could achieve p999 latency of less than 20ms, while 5 concurrent requests reduced latency to below 5ms, albeit with lower throughput. These findings are being used to define pre-built configurations for different environments, simplifying setup for users, while future posts will explore performance tuning in languages like C# that can leverage multiple CPU cores more effectively.