Why we rewrote WAL-G for Postgres backups in Rust: Meet WAL-RUS

At ClickHouse Cloud, ensuring reliable backups for PostgreSQL involves managing write-ahead log (WAL) archival to maintain data durability and recoverability. While WAL-G, a Go-based tool, has been effective in this role, its garbage-collected nature leads to unpredictable memory usage, which poses challenges in resource-constrained environments. To address this, ClickHouse developed WAL-RUS, an open-source Rust-based tool that offers predictable memory efficiency and compatibility with WAL-G. WAL-RUS utilizes Rust's explicit memory management and a daemonized architecture to minimize unnecessary buffering and maintain stable memory profiles, reducing memory consumption by over 70% compared to WAL-G. Benchmarks indicate that WAL-RUS achieves comparable archival throughput while offering a more stable resource footprint. This makes capacity planning simpler and aligns with the needs of modern, scalable applications. WAL-RUS remains fully compatible with existing WAL-G configurations and archives, simplifying migration for current users, and it introduces support for Postgres 17's WAL summaries for incremental backups. The tool is open-source and aims to become the default backup and WAL archival mechanism for ClickHouse Cloud's managed Postgres offering, inviting community feedback and contributions.