ScyllaDB’s Change Data Capture Library for Rust

ScyllaDB, in collaboration with the University of Warsaw, has developed a new Rust library for consuming Change Data Capture (CDC) logs, which is faster and uses less CPU than its Go and Java counterparts. This initiative involved a group of four students who, under the guidance of ScyllaDB engineers, created a library that enables real-time data processing and integration with other systems, essential for immediate reactions to database modifications. The library, written in pure Rust using the Scylla Rust Driver and Tokio, features a callback-based interface, automatic error retries, and optional checkpointing for progress saving. Benchmarks indicate that the Rust library outperforms the Java driver and is at least as fast as the Go library while consuming significantly less CPU. This project, forming the basis of the students' Bachelor's thesis, demonstrates Rust's potential in building efficient applications connected to ScyllaDB clusters, highlighting its user-friendly interface through example applications and benchmarks. The library's development, alongside ScyllaDB's ongoing support for Rust, underscores a shift toward more efficient and scalable solutions in data processing.