Eliminating Volatile Latencies: Inside Rakuten’s Cassandra Migration

Rakuten, a leading Japanese online shopping platform, faced challenges with volatile latencies while using Apache Cassandra for their catalog system, which handles massive data from numerous vendors and requires real-time updates for optimal shopping experiences. To address these issues, Rakuten migrated to ScyllaDB, which offered the same horizontal scalability but with more consistent performance due to its C++ base and Seastar framework, reducing kernel-level contention and improving parallelism. The transition led to a significant improvement in their data processing capabilities, including a 30% to 35% increase in product feed ingestion rates and a 2.5 to 5 times enhancement in publishing enriched data to partners. This migration also allowed Rakuten to reduce their cluster size from 21 Cassandra nodes to just 6 ScyllaDB nodes, resulting in lower total cost of ownership due to hardware savings and easier cluster administration. ScyllaDB's Incremental Compaction Strategy and shard-aware drivers further contributed to these performance gains, providing Rakuten with a more efficient and reliable system to support their vast and rapidly growing catalog.