The 5 Metrics that Predict Cache Outages

In building Momento, a production caching infrastructure, the team identified five critical metrics that predict potential cache outages. These include memory utilization and eviction rate, where a high eviction rate can alter cache behavior, leading to increased database load and latency issues. Connection churn, which focuses on the creation of new connections per second, can indicate problems with connection pooling that might destabilize the system. P999 latency, representing the slowest 0.1% of requests, serves as a crucial metric, as significant differences between P99 and P999 can reveal resource contention affecting user experience. Cache miss rate, rather than hit rate, provides insights into database load, as even slight increases can substantially impact backend performance. Finally, replication lag in systems using Valkey with replication serves as an early warning of network or capacity issues. The document emphasizes that while these metrics are essential, the true challenge lies in designing systems that automatically respond to them, ensuring stability and reducing the need for reactive interventions.