The Strategy Behind ReversingLabs’ Monster Scale Key-Value Migration

ReversingLabs successfully executed a massive migration of over 300 terabytes of data and 400 services from their internally-developed key-value database to ScyllaDB without any downtime. Initially limited by database options when they started in 2011, the company built their own efficient key-value store to handle their unique demands as a startup. However, as their system complexity grew, they opted for ScyllaDB due to its speed, resilience, and scalability. The migration strategy involved maintaining the existing database interface, making minimal changes to the data model, and leveraging Kafka for streaming data. The service migration was carefully staged, starting with APIs, followed by master and delivery services, ensuring continuity for API users. A significant challenge was adapting to ScyllaDB’s leaderless architecture, which necessitated developing a new distributed locking mechanism. The transition was seamless, resulting in improved system resiliency and high availability.