December 2024 Summaries

The ScyllaDB blog highlights the most popular posts from 2024, showcasing contributions from community members and engineers who delve into topics like database performance, caching, and scaling. Notable entries include a comparison between ScyllaDB and Memcached, insights into ScyllaDB's internal cache and new "tablets" data distribution architecture, and a debate on Rust vs. Zig programming languages. Other significant posts explore optimizing database performance with modern SSDs, the successful migration from PostgreSQL to ScyllaDB for faster query processing, and early performance benchmarks of Google Cloud's Z3 instances. The blog also revisits "timeless classics" from previous years, such as discussions on io_uring and eBPF in Linux, benchmarking MongoDB against ScyllaDB, and major migrations by companies like Disney+ Hotstar.

ScyllaDB is transitioning its ScyllaDB Enterprise product from a closed source to a source available license starting with the 2025.1 release, while maintaining the final open-source ScyllaDB OSS AGPL 6.2 release. This change is intended to unify the release streams, simplify development, and offer a more complete core product by incorporating previously peripheral functionalities into the main database, such as backup and restore, which eliminates the need for external management tools. The shift aims to enhance product manageability and functionality while offering a free tier of ScyllaDB Enterprise to the community, featuring capabilities like higher throughput, lower latency, and improved storage utilization. Despite potential concerns from open-source users, ScyllaDB believes this decision will allow the company to serve its customers better, streamline its development process, and continue advancing towards important milestones like Raft integration, tablet elasticity, and tiered storage.

Joan, a character in the book "Database Performance at Scale," navigates various database performance challenges after joining a company transitioning to a new distributed database system that maintains SQL's ACID guarantees. Initially tasked with creating a service for real-time database statistics, Joan faces issues when her chosen driver, a thin wrapper over legacy C code, causes errors. She resolves this by switching to an open-source driver backed by the database vendor. Her subsequent task involves fixing a misbehaving application that overloads the database due to client-side timeout misconfigurations and an overeager speculative retry policy. Through collaboration and meticulous debugging, Joan identifies and rectifies these issues, ultimately improving the system's performance. The narrative, interspersed with humor, conveys lessons about careful driver selection, balancing client and server-side timeouts, and the importance of observability tools in database management.

Digital Turbine, a key player in the mobile ad tech industry generating $500M annually, successfully migrated its workloads from AWS's DynamoDB to Google Cloud Platform (GCP) with the help of ScyllaDB's DynamoDB-compatible API, ScyllaDB Alternator, in less than a sprint. The migration was driven by the need to standardize on GCP following a series of acquisitions, along with issues related to cost and performance with DynamoDB, which required frequent scaling due to extensive read operations. Under the leadership of Joseph Shorter and Miles Ward, the migration was executed with minimal refactoring and risk, resulting in improved performance and reduced costs by approximately 20%. This strategic move not only aligned with Digital Turbine's growth plans but also positioned them to compete more effectively in the mobile ad tech space.

ScyllaDB 2024.2 introduces a significant upgrade with the implementation of "tablets," a new replication architecture enhancing scalability, elasticity, and operational efficiency, while reducing network costs and improving speed. Building on the Raft consensus protocol, this release enables dynamic data distribution and features such as file-based streaming for tablets, strongly consistent topology and authentication updates, and enhanced network compression using zstd. The update also includes improvements to the DynamoDB API (Alternator) with role-based access control and introduces a native nodetool to replace the Java-based version, streamlining operations further. The new tablets architecture allows faster scaling by dynamically distributing data across nodes based on size, which optimizes performance and supports mixed clusters with varying core counts. Additionally, maintenance mode has been introduced to facilitate easier node repairs without affecting client communications. Overall, ScyllaDB 2024.2 prioritizes elasticity, speed, and simplicity, providing significant advancements for enterprise users.