July 2020 Summaries

Stadia Maps was founded to provide aesthetically pleasing, fast-rendering maps at a lower cost than existing services, using open mapping programs and OpenStreetMap data. Initially reliant on CockroachDB, they switched to ScyllaDB due to CockroachDB's latency issues and inability to adjust consistency settings to their needs, which caused delays and errors in map rendering. ScyllaDB's eventually consistent global distribution capabilities allowed Stadia Maps to meet their end-to-end latency requirement of 250 ms, significantly improving performance by reducing response times to around 500 ms for p99. This switch facilitated efficient data handling and reduced server resource demands by balancing CPU, RAM, and storage needs between map rendering engines and ScyllaDB. The migration to ScyllaDB, which took just a month, has allowed Stadia Maps to optimize their systems continually, ensuring swift and reliable map delivery worldwide.

ScyllaDB Open Source 4.1.2 is a bugfix release for the ScyllaDB 4.1 stable branch, ensuring backward compatibility and supporting rolling upgrades. It addresses a variety of issues from previous versions, such as fixing the concurrency control in view update generation, resolving a crash issue when querying the CDC log with an incorrect stream_id, and improving user experience by fixing serialization errors and RAID disk prompts. Additionally, the release introduces an option to start Alternator with HTTPS in Docker and resolves issues related to LWT operations, memory management, and stability concerns linked to unclosed partitions and oversized allocations. These updates aim to enhance the overall stability, performance, and usability of ScyllaDB, while also including fixes from the earlier 4.1.1 release.

Change Data Capture (CDC) in ScyllaDB is a powerful feature that allows users to track data updates by recording modifications in a database table, enabling asynchronous consumption of these changes. Unlike Apache Cassandra's implementation, ScyllaDB's CDC is more accessible, as it is just another CQL table, allowing for easy integration and consumption using standard CQL queries. Users can customize the level of detail captured, such as pre-image, post-image, and deltas of changes, with options to limit storage overhead through TTL settings. ScyllaDB's CDC offers use cases like fraud detection, data streaming, and data replication, while ensuring data consistency and distribution align with the original data to minimize system impact. Despite the performance overhead of additional reads required for pre and post-image data, the CDC implementation is designed to be robust and easy to integrate, providing deduplicated and ordered data changes that can be consumed in various ways, including integration with tools like Kafka.

ScyllaDB Open Source 4.0.4 is a bugfix release for the ScyllaDB 4.0 stable branch, offering backward compatibility and support for rolling upgrades, although an upgrade to the latest ScyllaDB 4.1 is recommended. The release addresses several issues, including fixing problems with CQL where default TTLs are reset and unrecognized entity exceptions occur, as well as storage issues like the SSTable upgrade's increased disk space requirement. It also resolves user experience errors, such as a serialization error in scylla-housekeeping and traceback occurrences in scylla_setup, and enhances stability by addressing issues like unbounded internal concurrency in counter write operations and incorrect results or crashes during reverse queries with multiple IN restrictions. Performance improvements include addressing the needless retention of tombstones in local system tables, which could lead to read timeouts.

ScyllaDB Virtual Workshops are designed to enhance participants' understanding of ScyllaDB, a NoSQL database, through bi-monthly sessions that include demonstrations, design principles, and Q&A discussions. The workshops focus on practical deployment using Docker, addressing real-world big data use cases, and are ideal for professionals with basic Linux/Docker experience and familiarity with NoSQL. Attendees, including database architects and developers, can benefit from insights on monitoring, migration strategies from systems like Apache Cassandra, and optimizing system performance with ScyllaDB's tools such as ScyllaDB Manager and Monitoring Stack. The sessions also cover compatibility with Cassandra drivers, the use of shard-aware drivers for performance improvements, and considerations for database scaling and node reduction. Additionally, participants can learn about the integration of ScyllaDB with Kubernetes, Prometheus for metrics exposure, and the use of directly-connected SSDs for optimal throughput. Upcoming workshops and on-demand webinars are available for continued learning and engagement with ScyllaDB's community.

ScyllaDB and DataStax Enterprise both originate from Apache Cassandra, but ScyllaDB distinguishes itself by being a complete rewrite in C++, optimized for modern hardware and free from legacy Java code constraints, such as JVM Garbage Collection pauses. This architectural choice confers advantages in performance and reliability, making ScyllaDB a compelling alternative for those considering database solutions. A whitepaper detailing the reasons to switch from DataStax to ScyllaDB outlines key benefits such as price-performance, operational simplicity, maintainability, and ease of migration, alongside a feature comparison of the two databases. For individuals interested in exploring these differences further, ScyllaDB offers resources like webinars, a Slack channel for community interaction, and educational courses through ScyllaDB University.

Lightweight transactions (LWTs) in ScyllaDB offer a more efficient alternative to those in Apache Cassandra, reducing the number of required round trips from four to three, thereby improving performance and reducing timeout exceptions. While Cassandra's LWTs have faced challenges and limitations, ScyllaDB aims to overcome these by providing a more robust implementation that emphasizes performance, observability, and versatility. LWTs in ScyllaDB use the Paxos consensus protocol to ensure atomic compare-and-set operations, allowing conditional updates that do not lock the database. This feature is particularly useful in scenarios requiring transactional integrity, such as financial applications where the system handles concurrent requests and potential failures. ScyllaDB's approach to LWTs includes steps to manage contention and retries, control the growth of system tables, and maintain durability, all while ensuring compatibility with existing Cassandra semantics. The article also discusses the implementation of a scalable banking application using LWTs, illustrating the practical application and challenges of this technology, such as handling high contention workloads and ensuring data consistency across distributed nodes.

ScyllaDB Enterprise 2019.1.10, a new production-ready patch release, has been announced, focusing on enhancing stability and addressing various bug fixes to boost the performance of the high-performance enterprise NoSQL database for real-time big data workloads. This release advises ScyllaDB Enterprise customers to upgrade in coordination with the support team to benefit from improved performance and stability. Key issues addressed include the elimination of unnecessary tombstones in local system tables, adjustments to compaction settings affecting default time-to-live, freezing issues related to decimal-to-float casting, problems with large partition or clustering keys, and incorrect materialized view updates with future partition tombstones.

ScyllaDB Manager 2.1.1 has been released as a patch update to the stable 2.1 branch, aimed at improving the centralized cluster administration and recurrent tasks automation features of ScyllaDB Manager. This update specifically addresses a bug in the intensity flag, which was responsible for controlling the repair speed but was causing only one shard to be repaired at a time instead of a percentage of shards in parallel, leading to slower repairs than anticipated. The release does not affect the Manager Agent, and users are encouraged to coordinate with the ScyllaDB support team for the upgrade. The update ensures that the intensity flag is now correctly displayed in task progress, enhancing the user experience.

ScyllaDB Manager is an essential tool for automating backup tasks in ScyllaDB, a high-performance, distributed database system. It simplifies the process of managing backups by allowing users to configure various operations such as data selection, deduplication, retention, and throttling, which minimizes disruptions to workflows. The introduction of ScyllaDB Manager Agent in version 2.0 has improved backup efficiency, with version 2.1 further enhancing performance by reducing backup time and stabilizing memory consumption. Users can easily set up backup tasks that run on schedules and store data in specified locations like S3 buckets, with features like snapshot tagging and deduplication ensuring optimal use of storage and bandwidth. While the restoration process currently requires manual intervention, future releases aim to automate it, complementing the existing capabilities of ScyllaDB Manager, which also include cluster repairs and health checks.