Scaling Beyond Vertical Limits: Sharded Data Migrations at Webflow

Webflow's data architecture underwent significant changes due to increased scale and reliability demands, transitioning from a single vertically-scaled database to a horizontally-sharded system. Initially, the platform relied on a single database for core workloads, but as data volume and throughput grew, vertical scaling reached its limits, prompting a migration to a sharded architecture. A trickle migration strategy was employed to ensure product stability and zero downtime, incorporating an adapter layer to handle data queries seamlessly across sharded and non-sharded databases. This approach allowed for incremental validation and minimized operational risks, ensuring data consistency through a Redis-backed locking mechanism during migrations. The migration resulted in a substantial reduction of Webflow's primary database size by over 50%, improving performance and reducing operational risks, while establishing a scalable framework for future data migrations and horizontal sharding, enhancing the platform's capacity for continued growth.