Saving another 100TB of RAM with math (and Rust)

Cloudflare reduced the global memory use of its Pingora Backend Router by more than 100 TB by optimizing its consistent-hashing implementation, pingora-ketama, which routes cacheable requests to servers while maintaining stable cache placement. Consistent hashing uses virtual hash points to distribute workloads evenly and can weight servers by storage capacity, but Cloudflare’s feature-specific routing requirements created dozens of hash rings containing large numbers of points, consuming up to 6 GB of memory in some instances. Engineers first compacted each stored hash point from eight to six bytes by replacing a wasteful 32-bit server index with a 16-bit index and avoiding Rust structure-alignment padding, cutting hash-ring memory by 25%. Mathematical analysis and simulations then showed that the existing weighted configuration used far more hashes than necessary, with diminishing accuracy benefits and increasing collision risks from 32-bit hashes, allowing the team to reduce hashes per server by 90% without meaningful routing degradation. To prevent widespread cache invalidation and increased origin traffic, Cloudflare ran old and new rings simultaneously and migrated traffic gradually by data center while monitoring routing, errors, cache behavior, memory, and origin load. The resulting v2 ring, available in the pingora-ketama crate, also includes compact storage, faster sorting, configurable hash counts, and support for controlled parallel migrations.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.