Replicating a PostgreSQL Cluster to Redshift with AWS DMS

Over the past year, RevenueCat has developed a new data platform to enhance data-driven features, utilizing AWS Database Migration Service (DMS) to replicate data from a PostgreSQL database to a Redshift instance for analytics purposes. Given their PostgreSQL database processes over 800 changes per second, finding a reliable replication mechanism was crucial, leading them to choose AWS DMS for its security, ease of use, and cost-effectiveness. Despite its advantages, such as automatic DDL propagation and comprehensive documentation, the team encountered challenges like memory leaks, missing changes, and complex configuration requirements. These issues prompted the creation of internal tools to manage and rectify replication problems, including a daemon for detecting missing updates and handling Redshift maintenance windows. After a year of refining their approach and leveraging both AWS updates and their internal solutions, RevenueCat successfully integrated DMS into their production environment, facilitating the transfer of data from PostgreSQL to Redshift while acknowledging the tool's potential yet immature aspects.