Company
Date Published
Author
Kim Kaluba
Word count
742
Language
English
Hacker News points
None

Summary

Change Data Capture (CDC) is a proven data integration pattern that helps maintain consistency and functionality across all systems that rely on data. Many database systems have CDC capabilities, but these often hold little value to the systems outside of that database. There are four common methods to perform CDC: Date Modified, Diff, Log-Based, and Triggers, each with its own challenges. The Date Modified method is flawed by requiring deleted records to be kept, while the Diff method requires a large compute environment. The Log-Based method can be difficult to understand due to lack of standards on logging data, and the Trigger method increases overhead to record changes. CData Sync leverages CDC to track every change applied to a table and records those changes via a shadow history table, solving many of the challenges with trigger-based CDC by capturing full rows of data and selecting from the history view instead of interacting directly with the source table.