How Numberly Replaced Kafka with a Rust-Based ScyllaDB Shard-Aware Application

Numberly's Omnichannel Delivery team replaced their messaging infrastructure, previously reliant on Kafka, with a Rust-based ScyllaDB shard-aware application to streamline message processing and improve reliability. The new system consolidates message handling into a single platform, allowing for deterministic workload distribution and reduced dependency on multiple technologies, enhancing reliability and idempotence. The change required significant architectural adjustments, including the use of ScyllaDB's shard-per-core architecture to align data processing with CPU cores, effectively turning ScyllaDB into a distributed cache and ordered-data bus. The team chose Rust for its efficient, low-level capabilities, contributing to the ScyllaDB Rust driver to facilitate this transition. The implementation faced challenges, such as ensuring idempotence and optimizing memory usage, but ultimately led to a highly efficient and scalable system that earned the ScyllaDB Innovation Award for Top Technical Achievement.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.