July 2023 Summaries

The text discusses the pros and cons of vertical scaling, specifically focusing on a multi-threaded system called Dragonfly. While some technical leaders may initially be skeptical about the benefits of this approach due to its single-node architecture, the author argues that it can address production issues and lower costs. Vertical scaling is compared with horizontal scaling, which is often considered infinite but has limitations such as decreased efficiency when dealing with massive instance sizes and increased vulnerability to failure in stateful systems. The text also highlights some hidden hurdles of horizontal scaling, including the fragility of bigger clusters, the need for larger margins of over-provisioning for smaller nodes, and cloud over-commit issues that can reduce throughput per node. Finally, the author presents Dragonfly as a cost-efficient and well-maintained source-available technology that provides vertical scale out of the box, offering significant potential to reduce infrastructure costs compared to alternatives.

Dragonfly is an in-memory datastore that can act as a drop-in replacement for Redis, offering hardware efficiency and reduced architectural complexity. Migration from Redis to Dragonfly can be achieved using techniques such as Snapshot and Restore, Replication, or custom solutions. However, these methods may involve downtime. To perform zero-downtime migration, Redis Sentinel can be utilized. By replicating data from Redis to Dragonfly and then failing over to Dragonfly using Redis Sentinel, the failover is automatic and without manual steps of stopping or reconfiguring the application.