January 2020 Summaries

ScyllaDB Manager 2.0.1 has been released as a production-ready version for both ScyllaDB Enterprise and Open-Source customers, offering significant enhancements such as recurrent backups to AWS S3 and the introduction of the ScyllaDB Manager Agent. The recurrent backup feature allows users to schedule periodic or ad-hoc backups with customizable settings for bandwidth, parallelism, and timing, while also handling SSTable deduplication and providing progress reports. The new Manager Agent is designed to streamline communication between the Manager Server and ScyllaDB nodes over HTTPS, eliminating the need for SSH connectivity and minimizing performance impact through CPU pinning and Systemd slices. Additionally, ScyllaDB Manager now supports IPv6 for communication between the Manager and ScyllaDB nodes, aligning with the growing adoption of IPv6 due to the depletion of IPv4 addresses. The release also includes updates to the ScyllaDB Monitoring Stack with a new Manager 2.0 dashboard to track metrics effectively.

ScyllaDB Cloud is a fully managed NoSQL Database as a Service (DBaaS) offered by ScyllaDB, running on Amazon Web Services (AWS) I3 instances to deliver high-performance real-time applications by leveraging ScyllaDB Enterprise, which is API compatible with Apache Cassandra but offers significantly better performance. Recent updates include the ability for users to deploy ScyllaDB Cloud databases within their own AWS accounts, allowing them to manage resources with their own pricing and policies while still benefiting from the fully maintained aspects of ScyllaDB Cloud. Additionally, ScyllaDB Cloud has introduced support for extracting database metrics in Prometheus format, facilitating integration with centralized monitoring systems. The service now employs ScyllaDB Enterprise version 2019.1.4, ensuring users receive the latest stable updates automatically.

Fanatics, a leading online sports apparel company, has transitioned its database infrastructure from Apache Cassandra to ScyllaDB to address scalability and performance challenges. Founded in 1995, Fanatics has grown significantly, now valued at $4.5 billion with $2.6 billion in annual merchandise sales, and operates numerous ecommerce sites for various sports leagues. Initially moving to the cloud in 2015, Fanatics faced issues with Cassandra, such as node sprawl and frequent garbage collection pauses, leading to increased AWS costs and maintenance efforts. In 2019, the company successfully migrated from 43 Cassandra nodes to 6 ScyllaDB nodes, significantly reducing operational costs and improving performance, particularly during high-traffic events. This transition also involved developing custom management tools for cluster automation and autoscaling on AWS, enhancing both efficiency and reliability. The migration resulted in smoother operations with zero timeouts during peak traffic, making both customers and internal teams more satisfied with the platform's real-time responsiveness.

ScyllaDB Open Source 3.2 introduces significant new features and enhancements, including support for IPv6, CQL GROUP BY and LIKE operations, open range deletions, and Zstandard compression, alongside stability improvements. This release also debuts experimental features such as Lightweight Transactions using the Paxos consensus algorithm, Change Data Capture for monitoring data changes, and an Amazon DynamoDB-compatible API called "Alternator," reflecting ScyllaDB's ongoing development efforts to provide a high-performance, Apache-Cassandra-compatible NoSQL database with low latencies. The update marks a shift in support, focusing on the latest releases of ScyllaDB Open Source 3.2 and 3.1, while discontinuing support for version 3.0.

The blog post discusses the limitations of the Size-Tiered Compaction Strategy (STCS) used in databases like ScyllaDB, which requires maintaining at least 50% of disk space free to ensure smooth operations during data compaction, leading to significant storage inefficiency and increased costs. To address this, the Incremental Compaction Strategy (ICS) has been introduced, which combines elements of both Size-Tiered and Leveled compaction strategies to drastically reduce temporary space overhead during compaction. ICS achieves this by breaking down large SSTables into smaller fragments, allowing for more efficient space usage without increasing read or write amplification. The strategy is available in ScyllaDB Enterprise releases from version 2019.1.4 onwards and allows for greater disk utilization, enabling users to use up to 90% of available disk space without the risk of running out of space during compactions. The post recommends ScyllaDB Enterprise users transition from STCS to ICS to maximize disk usage and minimize costs, while maintaining performance efficiency.

Comcast's migration from Apache Cassandra to ScyllaDB for its Xfinity X1 platform has resulted in significantly improved performance, particularly in reducing long-tail latency, prompting the removal of its caching layer for the user interface. During the ScyllaDB Summit 2019, Philip Zimich, Senior Director of Software Development & Engineering at Comcast, detailed the transition from Oracle to NoSQL via Apache Cassandra, then to ScyllaDB, and outlined a roadmap through 2022. This transition has led to a substantial reduction in node count from 962 Cassandra nodes to 78 ScyllaDB nodes, while ensuring scalability for more than 31 million set-top boxes and second-screen devices across 15 million households. The switch is expected to save over 60% of Cassandra operating costs, and the X1 Scheduler, which processes over 2 billion RESTful calls daily, employs multiple datastore technologies, including MongoDB and Elasticsearch, alongside ScyllaDB.

In 2019, ScyllaDB experienced significant growth and engagement within its community, as reflected in the popularity of its blog posts, which highlighted several key developments and innovations. Among these, the introduction of ScyllaDB Open Source 3.0 topped the list, showcasing advanced features like production-ready materialized views and global secondary indexes that placed it on par with, and even surpassing, Apache Cassandra. Other notable topics included the ScyllaDB Alternator API, which provided an open-source, DynamoDB-compatible solution, and the use of SystemD slices for optimizing performance on Linux systems. The blog also delved into collaborative uses of ScyllaDB with technologies like Elasticsearch and JanusGraph, and addressed industry trends such as the potential rise of Arm-based servers in the market, highlighting the company's commitment to innovation and responsiveness to user interests.

ScyllaDB Enterprise 2019.1.4 has been released, bringing enhancements focused on stability, storage efficiency, and IPv6 support for high-performance real-time big data workloads. The update introduces the Incremental Compaction Strategy (ICS), which addresses the temporary disk usage issue of the previous Size Tiered Compaction Strategy (STCS) by breaking large SSTables into smaller chunks, allowing for more efficient disk usage and potentially reducing storage costs by over a third. This version also supports IPv6 for client-to-node and node-to-node communication, aligning with the depletion of IPv4 addresses. The release addresses several stability issues, including core dumps and handling of end-of-disk space states, ensuring a more robust user experience.

ScyllaDB has released the ScyllaDB Monitoring Stack 3.1, an open-source tool designed to monitor ScyllaDB Enterprise and Open Source versions using Prometheus and Grafana. This version supports ScyllaDB Open Source 2.3, 3.0, 3.1, and the upcoming 3.2, as well as ScyllaDB Enterprise 2018.x and 2019.x, and ScyllaDB Manager 1.4.x and 2.0. New features include dashboards for ScyllaDB Open Source Version 3.2 and ScyllaDB Manager Version 2.0, an ad hoc filter for flexible graph filtering, a report-a-problem button for quick issue reporting, annotations for server restarts, and CQL optimization warnings for consistency level hazards. There's also an experimental integration with ScyllaDB Manager Consul, allowing cluster server information to be directly obtained from ScyllaDB Manager. Bug fixes and maintenance updates have also been implemented, alongside migrations and optimizations such as the conversion of single-stat panels to gauges and adjustments to OS metrics displays.