December 2023 Summaries

PeerDB introduces Change Data Capture (CDC) from Postgres 16 Read Replicas, offering a fast and simple experience for replicating data from Postgres to various destinations. This feature eliminates risks on the Primary Postgres Database by enabling CDC/replication from Read Replicas instead of Primaries. PeerDB's learnings while building this feature include: 1. CREATE_REPLICATION_SLOT works as expected on Read Replicas. 2. SNAPSHOT can be created and used on Read Replicas. 3. START_REPLICATION worked as expected on Read Replicas. 4. Publications cannot be created on the Read Replica, requiring manual creation on the primary. 5. WAL control functions cannot run on Read Replicas, necessitating a conditional query using pg_last_wal_receive_lsn() for replicas. To try out this new feature, users can fill out the Request for Access form on PeerDB's website or visit their GitHub repository to Get Started.

The new open-source tool, pgslot-notify-bot, has been launched to help database administrators monitor and manage the growth of PostgreSQL replication slots. This tool provides monitoring features, instant Slack alerts for designated channels, easy setup with an .env file or docker-compose.yaml, and adjustable thresholds. Future plans include evolving the bot to monitor other common issues in PostgreSQL data replication. Users are encouraged to try out the tool, report any issues, contribute to its development, and leave a star if they find it useful.

Logical decoding is a mechanism in Postgres that enables users to stream changes as a sequence of logical operations. It's useful for applications that need to keep an external data store synchronized with Postgres, such as replicating data in Postgres to Data Warehouses for analytics. Monitoring slot activity and its growth can help detect if the consumer application is lagging or halted. Large transactions can lead to Slot growth, but this can be avoided by using the START_REPLICATION command. PUBLICATIONs can filter logical decoding changes based on specific tables, schemas, types of changes, and even column values. The logical_decoding_work_mem setting controls how much memory a logical replication connection can use to decode changes. Postgres 16 allows creating replication slots on standby, offering benefits like easing the load on the primary server and eliminating the risk of replicating slot growth impacting the primary.