How to choose the right metadata store

The blog series by Rubrik explores their decision to transition from using Cassandra to CockroachDB as their metadata store for Rubrik CDM, highlighting the challenges faced with Cassandra such as uneven data distribution, inefficiencies of secondary indexes, out-of-memory crashes, and issues with deleted row resurrection. Seeking a solution that could offer high consistency, good performance under stress, and ease of maintenance, Rubrik developed a stateless translator called CQLProxy to facilitate the migration without altering application layers. The adoption of CockroachDB reduced operational overhead and simplified application logic, though challenges like clock skew and backpressure required innovative solutions like the development of Kronos, a custom time service, and additional application and database adjustments. Rubrik's experiences underscore the importance of choosing a metadata store that aligns with system architecture and operational requirements, offering insights into metadata management and distributed database optimization.