February 2018 Summaries

ScyllaDB Enterprise 2017.1.5 is a newly released, production-ready minor update to the ScyllaDB Enterprise 2017.1 branch, which is the latest stable version based on the open-source ScyllaDB 1.6. This release primarily addresses bug fixes, incorporating both enterprise-specific and backported fixes from upstream versions 1.7, 2.0, and 2.1, enhancing the database's capability to handle high-performance, real-time big data workloads. Customers are encouraged to upgrade to this version to benefit from these improvements and can access the update through customer-exclusive channels or a 30-day evaluation. A notable fix addresses the issue where ScyllaDB fails to start if the number of IO queues does not match the cpuset configuration.

ScyllaDB 2.1 represents a significant update to the open-source NoSQL database, known for its compatibility with Apache Cassandra and enhanced performance, offering new features and improvements. This release introduces the Time Window Compaction Strategy (TWCS) for more efficient handling of time series data, CQL additions like cast functions and support for the Duration data type, and enhancements in memory efficiency through optimized storage of CompressionInfo.db files. The update also includes transitional authentication features for seamless access control, a new REST API for aborting repairs, and experimental secondary indexing capabilities. Additionally, ScyllaDB's monitoring stack has been updated with a new dashboard and several new metrics to better assess database performance. With critical bug fixes limited to versions 2.1 and 2.0, users are encouraged to upgrade from earlier versions to benefit from these advancements.

The Seastar framework, known for providing high-throughput I/O and networking capabilities to ScyllaDB and other programs, has been accepted as a Google Summer of Code organization. This initiative offers students the opportunity to contribute to an open-source project during their summer break, working under the mentorship of experienced engineers while earning a stipend upon meeting project milestones. Interested students are encouraged to check their eligibility and explore potential project ideas, with the application period running from March 12th to March 27th. Additional information and guidance can be obtained through the Seastar Developer’s Mailing List, and more details about the framework can be found on its official website.

In the blog post, the author describes a scenario involving the Mutant Monitoring System (MMS) where Division 3's mutant data centers face cyber attacks, necessitating preparation for disaster scenarios. The post outlines a hands-on exercise using ScyllaDB to simulate a node failure and explains the importance of understanding consistency levels for data accessibility. With a replication factor of three, data remains accessible even if one node goes down, as the default consistency level, QUORUM, allows the system to operate with a majority of nodes. The exercise involves pausing a node, verifying data integrity, and exploring different consistency levels such as ONE and ALL, which dictate how many replicas must acknowledge operations. The post concludes with instructions on adding a new node, repairing the cluster using a nodetool command to ensure all nodes are synchronized, and preparing the system for future attacks while hinting at future analytics opportunities using Apache Zeppelin.

ScyllaDB 2.0.3, released by the ScyllaDB team on February 20, 2018, is a bugfix update for the ScyllaDB 2.0 stable branch, ensuring backward compatibility and supporting rolling upgrades. Users upgrading from version 1.7.x should consult the release notes, especially since older driver versions are no longer supported. Key issues addressed in this release include fixes for streaming blocking gossip, upgrade difficulties with CentOS dependencies, and problems with overlapping range deletes, among others. Additionally, the Amazon Machine Image (AMI) for ScyllaDB 2.0.3 now uses kernel version 4.9.77-31.58.amzn1.x86_64, which addresses critical vulnerabilities. Users are encouraged to report any issues encountered with this release.

Seastar provides a programming environment that simplifies multi-threaded programming by using a thread-per-core model, eliminating the need for locks and atomic variables, and instead offering a single facility for inter-core communication. This design allows for advanced optimizations in inter-core communications, particularly beneficial for scaling on many-core machines. Traditional inter-thread communication methods using locks face scaling challenges in data-intensive applications, whereas Seastar employs a shared-nothing, asynchronous model where data structures are owned by specific cores, eliminating the need for locks. The text explores various lock-free techniques and optimizations, such as single-producer/single-consumer queues and memory barriers, to improve efficiency and reduce complexity in inter-core communications. Seastar also implements strategies to manage sleeping threads and memory barriers, initially using signals and later transitioning to more efficient methods like the Linux system call membarrier() for better performance across different architectures, ultimately showcasing Seastar's ability to adapt to and benefit from operating system advancements.

GoCQL is a Go package created in 2014 to facilitate the connection between Go applications and ScyllaDB by efficiently managing data center discovery, connection pooling, and query distribution. It offers enhanced query tracing for troubleshooting and performance analysis, along with network traffic optimization through features like Frame Compression and automatic Type Conversion for safe data handling. An extension of GoCQL, GoCQLX, further enhances developer productivity by providing query builders, support for named parameters, and convenient functions for handling query results, all while maintaining query performance.

ScyllaDB Manager, introduced by Michał Matczuk, is a management system designed to automate maintenance tasks for ScyllaDB clusters, with its first release focusing on a managed repair feature. This feature automatically conducts repairs on clusters by utilizing ScyllaDB's unique thread-per-core architecture, aiming to optimize repair parallelism and reduce repair time without adding unnecessary load. The system simplifies management by automatically discovering cluster topology and scheduling repairs, while allowing users to customize schedules for specific tables or keyspaces. Users can monitor repair progress in real-time through a REST API or CLI tool, and the system ensures that repairs are resumed or retried in case of failures. ScyllaDB Manager operates as a centralized product, storing management data within the ScyllaDB database, and connects to cluster nodes via SSH, eliminating the need for JMX. It provides user interaction through a REST API or the command line tool, sctool, which facilitates cluster registration and repair management.

ScyllaDB Manager 1.0 is introduced as a tool for centralized administration and task automation for ScyllaDB Enterprise, providing significant improvements in managing ScyllaDB clusters. This release focuses on automating recurrent repairs, addressing previous challenges such as the lack of a centralized management system and scheduler, and the difficulty in suspending and resuming repair tasks. ScyllaDB Manager consists of a Go-based server, a CLI tool named sctool, and a highly available backend for state persistence. It interacts securely with ScyllaDB clusters, automatically learns their topology, and executes cluster-wide tasks efficiently. Designed to be stateless, ScyllaDB Manager recommends using a multi-node ScyllaDB cluster for high availability, although by default it installs a local instance of ScyllaDB as its backend. Through ScyllaDB Manager, users can initiate, schedule, and monitor management operations, with the ability to handle multiple clusters simultaneously, thereby becoming an integral part of ScyllaDB Enterprise management.

ScyllaDB Enterprise 2017.1.4 is a minor release aimed at addressing various bugs within the 2017.1 branch, which is based on the open-source ScyllaDB 1.6, incorporating fixes from later versions and specific enterprise-level corrections. This release resolves issues such as outdated node status reports following disk space errors, excessive memory allocation potentially causing latency, failure to destroy SSTables during streaming flush failures, memory leaks in the LSA zone reclaimer, node restart failures due to batch assertion errors, and memory pressure from Prometheus metrics pulling. ScyllaDB Enterprise customers are advised to upgrade to this version for enhanced stability and are encouraged to reach out to support for any assistance needed.

Numberly's Alexys Jacob shares his experience evaluating ScyllaDB for production, highlighting critical insights from his initial preparation phase with the ScyllaDB team. Originally using MongoDB, Numberly faced performance challenges due to its primary/secondary architecture's impact on write throughput and the complexity of setting up clusters. Jacob also considered Apache Cassandra but was concerned about Java-related issues like heap sizes and garbage collection. Opting for ScyllaDB, a C++-based NoSQL database, he found it more efficient for leveraging machine resources. His blog post offers valuable feedback on conducting a successful proof of concept (POC) and emphasizes the importance of clarity, monitoring, and understanding the distinction between POC and production environments. Jacob plans to share more technical details in the second installment of his series.