Company
Date Published
Author
Olivia Greene, Ahmed Saef Zamzam, Prabha Manepalli, Nathan Nam, Weifan Liang
Word count
1383
Language
English
Hacker News points
None

Summary

Confluent has released the Oracle CDC Source Connector, which allows organizations to reliably and cost-effectively implement continuous real-time syncs by offloading data from Oracle Database to Confluent. This connector can be used alongside other Confluent components like ksqlDB and sink connectors for modern data systems to enable key use cases such as data synchronization, real-time analytics, and data warehouse modernization. The connector uses Oracle LogMiner to read the database's redo log and requires supplemental logging with ALL columns for tables that are of interest or for the entire database. It supports multiple versions of Oracle Database and can start reading logs from a specific system change number (SCN) or timestamp. Confluent's Oracle CDC Source Connector can continuously monitor the original database and create an event stream in the cloud with a full snapshot of all of the original data and all of the subsequent changes to data in the database, as they occur and in the same order. The connector can be used to write change events specific to tables in the Oracle Database to Kafka topics for further processing or analysis. Confluent has also released a fully managed BigQuery Sink Connector that can consume these event streams and apply those same changes to the BigQuery data warehouse, providing a working data pipeline from an Oracle database through to BigQuery.