Why Cache Data? [Latency Book Excerpt]

Latency is a critical concern at ScyllaDB, prompting the exploration of caching as a method to reduce latency in applications and systems. Pekka Enberg's book "Latency" addresses the complexities of latency by revealing techniques often hidden in developer folklore and explores practical applications of these techniques, including caching. Caching is recommended for systems that do not require complex queries or transactions and face compute or storage constraints, as it can significantly reduce access time by storing temporary data copies. However, it involves trade-offs such as data freshness and storage needs. Effective caching is measured by the cache hit-to-miss ratio, with a high hit ratio indicating better performance. Cache eviction policies are essential for managing limited storage space, determining which data remains in the cache, and ensuring data consistency when source data changes. Caching strategies vary in how they handle cache misses, influencing latency and complexity. Despite its challenges, caching remains a valuable technique for latency reduction, particularly when system changes are not feasible.