February 2019 Summaries

ScyllaDB University offers free, self-paced online training designed to help users master the ScyllaDB NoSQL database, renowned for its speed, self-optimization, and high availability as an Apache Cassandra replacement. The initial course, ScyllaDB Essentials, provides foundational knowledge of NoSQL databases and ScyllaDB's features, including installation, architecture, and configuration for high availability. The platform emphasizes a blend of theoretical and practical learning, incorporating quizzes and hands-on labs to enhance comprehension and retention. New courses are under development, covering advanced topics like ScyllaDB Administration, data modeling, and troubleshooting, alongside certification paths and user interface improvements. Participants can earn points through quizzes and lessons to compete in the ScyllaDB University Sea Monster Honor Roll Challenge, with incentives such as ScyllaDB merchandise for top performers.

Gleb Natapov's blog post explores the challenges of maintaining low latency in Big Data environments, emphasizing the need for constant technological evolution, akin to the Red Queen syndrome from evolutionary biology. As storage technologies advance from HDDs to SSDs and now NVMe drives, the industry must continually adapt to keep pace, with NVMe drives becoming the norm and SATA SSDs viewed as legacy. However, even NVMe can experience high latencies due to factors like SSD garbage collection and concurrent data operations, highlighting the benefits of in-memory solutions for latency-sensitive applications. To address these challenges, ScyllaDB has introduced an in-memory option in its enterprise version, which can significantly reduce latency for certain workloads without needing to rely on disk storage. The post also discusses the trade-offs between using in-memory and regular on-disk tables, depending on an application’s tolerance for latency variability, and provides guidance on configuring ScyllaDB to optimize performance.

ScyllaDB Open Source 2.3.3 is a bugfix release from the ScyllaDB team, maintaining backward compatibility and supporting rolling upgrades, although users are encouraged to upgrade to the latest stable release, 3.0. This release addresses several issues, including a race condition between the "nodetool snapshot" command and running compactions leading to errors, a problem where a bootstrapping node fails to wait for the schema before joining the ring, and an issue with ScyllaDB rejecting certain SSTables created by earlier versions of Cassandra. Additionally, it resolves rare instances where the commit log replay fails and where ScyllaDB unexpectedly exits during service stop.

ValuStor, introduced by Derek Ramsey at the ScyllaDB Summit 2018, is a NoSQL memory cache and persistent database built on ScyllaDB, designed as an alternative to memcached for handling key-value and document store data models. Developed by Sensaphone, a company specializing in remote monitoring solutions for the Industrial Internet of Things, ValuStor addresses the limitations of memcached, such as lack of persistence, encryption, and failover capabilities, by utilizing ScyllaDB's architectural advantages like performance optimization, automatic adaptive consistency, and robust fault tolerance. Released as open-source software under the MIT license, ValuStor is implemented with a single header-only database abstraction layer comprising a ScyllaDB backend, ValuStor Client, and Cassandra driver, offering enhanced security features, including TLS support and client authentication. This architecture allows ValuStor to scale efficiently, ensuring high availability and reliability, while enabling seamless integration with user applications through minimal API functions and native data type support. ValuStor's roadmap includes further enhancements such as SWIG bindings for multi-language support, improved command-line interfaces, and advanced feature exposure for processing multiple requests efficiently, making it a compelling choice for organizations seeking scalable, secure, and performant data solutions.

ScyllaDB Enterprise 2018.1.10 is a minor release aimed at resolving bugs in the 2018.1 branch, with its most notable improvement being the prevention of reactor stalls during MemTable merges into the cache, a feature backported from the open-source version. This update addresses several issues, including the erroneous rejection of SSTables created by Cassandra 2.1, the disabling of TLS1.0 by default for enhanced security, and rare instances of commit log replay failures and out-of-memory errors under heavy load. Additionally, ScyllaDB Enterprise 2018.1.10 introduces a new option to select the server NIC during setup, while tackling a bug where the setup script fails on Ubuntu 16.04 when the NIC isn't eth0. Current ScyllaDB Enterprise users are encouraged to upgrade in coordination with the support team to benefit from these enhancements.

ScyllaDB faced challenges in shipping Python scripts across various enterprise environments due to module dependencies, Python version conflicts, and stringent security policies. While considering alternatives like rewriting scripts in C++ or using tools like Cython and PyInstaller, ScyllaDB ultimately developed a unique solution: a relocatable Python3 interpreter. This interpreter is bundled with necessary libraries, enabling it to run on any Linux distribution without relying on system libraries. The approach ensures compatibility, maintains access to source code, and allows for script extensibility by including pip for installing additional packages. This method not only simplifies distribution but also addresses execution-time dependencies effectively, offering a robust solution tailored to their needs, although its broader applicability remains to be explored.

ScyllaDB users frequently share their experiences and innovative solutions made possible by the database technology, with many highlighting its ease of use and efficiency. At the ScyllaDB Summit, users like Derek Ramsey from Sensaphone and Murukesh Muhanan from Yahoo! Japan praised its performance, noting that ScyllaDB can handle workloads with significantly fewer nodes compared to Cassandra. The absence of a JVM, as pointed out by Keith Lohnes from IBM, eliminates certain issues, making it easier to configure for systems similar to Cassandra or DynamoDB. The company actively engages with its users through various channels, including a dedicated Slack channel, Zoom meetings, and a user conference, where they gather insights on how to improve their offerings. Video interviews and case studies featuring user stories are available on ScyllaDB's website, showcasing diverse applications ranging from social apps to IoT systems, as users like Alexys Jacob from Numberly emphasize the importance of a trustworthy and efficient system with consistent low latencies.

ScyllaDB Open Source 3.0.3 is a bugfix release for the ScyllaDB Open Source 3.0 stable branch, ensuring backward compatibility and supporting rolling upgrades. This release addresses several issues, including the rejection of SSTables containing counters from Cassandra 2.0 and earlier, disabling TLS1.0 by default to enhance security, and fixing rare instances where commit log replay fails after unexpected node restarts. Additionally, it resolves problems with failed streaming operations and a rare race condition that could cause ScyllaDB to exit during schema updates and node restarts. Users are encouraged to report any problems they encounter with this release.

The ScyllaDB Migrator is a Spark-based application designed to facilitate the efficient migration of data from Cassandra to ScyllaDB, leveraging Spark's parallel processing capabilities to enhance performance. The process involves creating an identical schema in ScyllaDB, configuring applications for dual writes, snapshotting historical data, and ultimately decommissioning Cassandra. The migrator is resilient to failures, can resume operations using savepoint files, and supports timestamp preservation and column renaming. While it competes for resources with Cassandra, careful tuning and deployment strategies, such as disabling compaction, can optimize its performance, achieving transfer rates of up to 3.73GB per minute. The blog provides detailed guidance on deploying Spark, running the migrator, and monitoring the transfer process, emphasizing the importance of balancing parallelism and resource allocation.

ScyllaDB Manager 1.3.1 has been released, offering new features for ScyllaDB Open Source users, who can now automate maintenance tasks on clusters of up to five nodes, previously a capability reserved for ScyllaDB Enterprise users. This update introduces enhancements such as support for encrypted CQL communication and improved SSH stability, along with the availability of an official ScyllaDB Manager Docker image for easier deployment. The management system performs regular health checks, allowing users to monitor node status and control repairs on a per-shard basis, optimizing repair speed and performance. It also supports SSL/TLS encryption for secure communication and data storage, with customizable options for client-server encryption. The inclusion of a keepalive mechanism ensures stable connections over SSH, preventing disruptions that may occur through proxies. These features align with ScyllaDB's shard-per-core architecture, enhancing the efficiency and reliability of managing ScyllaDB clusters.

ScyllaDB Open Source 3.0.2 is a bugfix release for the ScyllaDB Open Source 3.0 stable branch, maintaining backward compatibility and supporting rolling upgrades. Key issues addressed in this update include the correction of incorrect results when using "ALLOW FILTERING" in CQL, improvements in Hinted Handoff to ensure updates are sent promptly, and adjustments to prevent CPU stalls caused by a large number of SSTables during streaming. Additionally, a rare issue causing ScyllaDB to exit during service stops has been resolved. The release is available via Docker, binary packages, and EC2 AMI, with support for seamless upgrades from previous versions.

Meshify, a company acquired by Hartford Steam Boiler and owned by Munich RE, leverages Internet of Things (IoT) technology to prevent industrial disasters by using sensors to monitor conditions like temperature and water presence, sending alerts when predefined thresholds are exceeded. At the ScyllaDB Summit 2018, Sam Kenkel, DevOps lead at Meshify, discussed their use of ScyllaDB's high-performance database to efficiently process time-series data from these sensors. Meshify prioritizes vendor neutrality and uses cloud services with drop-in replacements to ensure system reliability, employing a strategy akin to the "pets vs. cattle" analogy in cloud computing, where systems are designed for easy replacement and minimal human intervention during failures. By quickly replacing failed nodes and deploying new clusters for disaster recovery, Meshify fulfills the vision of Hartford Steam Boiler's founders to proactively avert disasters in real-time, which earned them the "Fastest Time to Production" award at the summit.