June 2020 Summaries

The Ask Me Anything (AMA) session with Avi Kivity, ScyllaDB's CTO, covered various technical aspects and future directions of ScyllaDB, an open-source database. Avi discussed the potential programming languages for ScyllaDB, expressing a preference for Rust over JavaScript and Go due to its precise control and lack of a garbage collector. He highlighted the current limitations in supporting plugins, plans for extending ScyllaDB to cloud platforms like Azure and GCP, and the approach to implementing DynamoDB-compatible transactions. The dynamic memory allocation method was favored over object pooling due to its adaptability to irregular workloads. Avi also touched upon the use of C++20 for its coroutine feature, compared ScyllaDB to databases like Aerospike and MongoDB in terms of scalability and performance, and discussed the potential of running ScyllaDB on Arm architectures, specifically Amazon's Graviton2, contingent on the availability of fast NVMe storage. Debugging strategies were also detailed, emphasizing tracing and metrics over heavy reliance on logs to troubleshoot field issues.

ScyllaDB Open Source 4.1.0 introduces several improvements and updates, focusing on stability, bug fixes, and expanded functionality of the Alternator API. The release adds support for Red Hat Enterprise Linux 8, CentOS 8, and Debian 10, while retiring support for version 3.3. Key features include enhanced write isolation modes, access to system tables via the Alternator REST API, and linearizable lightweight transactions. It also introduces security enhancements like SSL Certificate Hot Reloading and various stability improvements, including fixes for integer overflow and race conditions. Additionally, the update disables the online loading of SSTable files in the main directory and continues testing Change Data Capture (CDC) for future releases. Other notable updates encompass improvements in compaction, performance, and monitoring, along with experimental features like Lua-based User Defined Functions (UDFs).

In June 2020, ScyllaDB University introduced a range of new educational content aimed at enhancing users' skills in ScyllaDB and NoSQL databases across various programming languages, including Rust, Go, Python, Node.js, and C++. The platform offers a diverse array of lessons, from building applications and performing database queries to advanced operational management and data modeling. Notable courses cover topics such as using the new DynamoDB-compatible API, understanding Time to Live (TTL) for data expiration, and implementing different compaction strategies in ScyllaDB. ScyllaDB University provides an online, self-paced learning environment that includes practical examples, quizzes, and hands-on labs, all available for free. Users can earn certificates upon course completion, contributing to their professional development and credibility in the field of NoSQL databases.

ScyllaDB University offers a course on utilizing the DynamoDB-compatible API in ScyllaDB, known as Project Alternator, which is designed to provide an open-source alternative to Amazon's DynamoDB. This project enables ScyllaDB to support DynamoDB client code without modification and can be deployed across various environments, including on-premises and cloud platforms like AWS, Azure, and Google Cloud Platform. The course guides users through setting up a ScyllaDB cluster, performing basic operations like creating tables, and executing read and write queries using the Boto 3 SDK for Python. Alternator aims to offer cost savings, enhanced performance, and open-source flexibility compared to DynamoDB. The lesson covers consistency levels, data modeling similarities, and differences between DynamoDB and ScyllaDB, and hints at more advanced topics like region management and data encryption in future lessons.

ScyllaDB Manager 2.1, a tool for centralized cluster administration and automated tasks, has been released for both ScyllaDB Enterprise and open-source users, featuring improvements in backup, repair, and health check functionalities. The update enhances backup speed to AWS S3 and usability with new commands, as well as reliability in health checks by eliminating false alerts with improved CQL ping execution. Repair operations now allow for more granular control over speed, and other enhancements include better error logging, updated configurations, and monitoring capabilities with the ScyllaDB Monitoring Stack 3.3. The release is backward compatible with version 2.0, maintaining continuity with existing backup plans and requiring coordination with ScyllaDB support for upgrades.

ScyllaDB Enterprise 2019.1.9 is a patch release that emphasizes stability improvements and bug fixes for enhancing the high-performance capabilities of this NoSQL database for real-time big data workloads. Released by the ScyllaDB team, it is recommended that ScyllaDB Enterprise customers upgrade in collaboration with the support team to benefit from the updates. The release addresses various issues such as incorrect IPv6 addresses in nodetool status, updates to AWS enhanced networking instances, and several stability concerns involving node failures, authentication errors, and deadlocks in multishard_writer during node repairs. These improvements aim to ensure the reliability and efficiency of ScyllaDB Enterprise for handling critical data tasks.

Opera, a NASDAQ-traded company with a significant user base primarily in Africa, has adopted ScyllaDB to improve the synchronization of user data across its various browsers and platforms. The move from Apache Cassandra to ScyllaDB was driven by performance issues and operational challenges with Cassandra, such as high latencies and frequent crashes. Opera initially faced difficulties with Cassandra's garbage collection pauses and maintenance headaches, prompting the exploration of ScyllaDB's shard-per-core architecture, which promised enhanced performance and reliability. After testing and benchmarking, Opera found that ScyllaDB could handle three times more operations per second with significantly reduced latencies compared to Cassandra. This led to a migration process that initially involved testing with a select user group before a full-scale transition. The successful migration resulted in a more efficient system with reduced node requirements and faster synchronization times, ultimately providing a more seamless experience for Opera's users without the operational burdens previously encountered.

ScyllaDB has announced an enhancement to its open-source ScyllaDB Migrator, allowing users to import Apache Parquet files directly into ScyllaDB tables, leveraging Apache Spark's distributed execution model for parallel data insertion. Previously, the Migrator supported data loading from Cassandra or DynamoDB into ScyllaDB and migrations between ScyllaDB clusters. The new feature uses Spark's DataFrame abstraction to load Parquet files stored on AWS, requiring users to configure their source settings and execute the migration through a Spark cluster. This development is part of ScyllaDB’s broader plan to expand the Migrator's capabilities with more data source and target types, aiming to transform it into a versatile database-to-database migration tool.

Time to Live (TTL) is a feature in ScyllaDB that allows for automatic deletion of expired data, offering flexibility by enabling TTL settings at both the column and table levels. TTL is specified in seconds and can be applied when defining a table or using INSERT and UPDATE queries, with a default value of null meaning no expiration. This functionality allows for managing data longevity with ease, as TTL can be set for individual fields or entire rows, and can even be updated or removed as needed. An example demonstrates creating a table with a default TTL, inserting data with specific TTL values, and using queries to observe how TTL affects data retention. For further learning, ScyllaDB University offers a detailed lesson on TTL.