June 2018 Summaries

The Mutant Monitoring Web Console, a project undertaken by Division 3, is a Node.js application designed to manage and track mutants via a central interface linked with ScyllaDB. This web console allows users to view and update mutants' photos and details, track various attributes such as heat, telepathy, speed, and location, and utilize a load generator to automatically populate tracking data. Users can manage mutant information through simple HTTP requests handled by the console, which interacts with ScyllaDB to perform operations like uploading photos, retrieving catalog data, and updating mutant details. The blog post also guides users on accessing and setting up the console through Docker, enhancing ease of use and functionality for monitoring mutants effectively.

ScyllaDB 2.1.5, released by the ScyllaDB team on June 25, 2018, is a bugfix update in the 2.1 stable branch, ensuring backward compatibility and supporting rolling upgrades. This release introduces an updated ScyllaDB AMI based on CentOS 7.5, addressing its absence in the previous 2.1.4 version, while maintaining the same kernel version of 4.9.93-41.60.amzn1.x86_64 as before. Users are encouraged to try ScyllaDB using various deployment options, including Docker and EC2 AMI, and to report any encountered issues. The release emphasizes ease of use through a Test Drive feature that allows users to quickly set up a ScyllaDB cluster and evaluate its performance.

Cloud computing has become a staple in modern infrastructure, enabling the rapid deployment of virtual machines with minimal user awareness of the underlying hardware. This flexibility is further enhanced by the use of containers, which abstract applications from physical infrastructure. However, AWS's introduction of the i3.metal instance type, which provides direct hardware access, contrasts with this trend of increased abstraction. i3.metal, devoid of a hypervisor, allows for more efficient resource utilization, offering significant performance improvements over its virtualized counterpart, i3.16xlarge. The absence of virtualization overhead results in up to 31% faster write rates and significantly lower read latencies. These performance gains, particularly for I/O-intensive applications, make i3.metal an attractive option for running ScyllaDB on AWS, offering enhanced scalability and efficiency by leveraging the direct access to hardware. The study highlights how removing the virtualization layer can optimize resource use, suggesting that i3.metal is well-suited for real-time Big Data workloads.

Part 15 of the Mutant Monitoring System blog series explores using Java to store binary files, such as images, in ScyllaDB, a distributed database known for its fault protection and resiliency. The process involves adding a blob column to the catalog.mutant_data table within ScyllaDB to hold images of mutants, allowing Division 3 to view and share these images with law enforcement. The post details setting up the ScyllaDB cluster, modifying the database schema, and running a Java application within Docker to store and retrieve images, using the blob data type for binary storage. This development enhances the system's capabilities, providing a robust method for managing and sharing critical visual data and demonstrating the flexibility and potential of integrating file storage within a database ecosystem.

ScyllaDB Enterprise 2018.1.3 was released as a minor update to address critical data loss issues associated with the Leveled Compaction Strategy (LCS) in the 2018.1 branch of the NoSQL solution. The bug, which could cause data loss during actions such as full table scans, decommissioning, adding nodes, and repairs, was traced back to a low-level optimization introduced in the 2.1 release and only surfaced during 2.2 testing. Users employing LCS are advised to upgrade immediately to mitigate risks, and those with relevant backups should contact support for restoration procedures. The issue was not identified earlier due to the ScyllaDB cluster test suite incorrectly attributing it to disruptor activity. Efforts are underway to enhance the test suite's error detection capabilities, and a comprehensive root cause analysis will be published. Additionally, the update resolves other issues, such as redundant requests to remote data centers and errors during node shutdowns when using TLS connections.

ScyllaDB Open Source 2.1.4 has been released as a bugfix update to the 2.1 stable branch, addressing a critical issue related to data loss when using the Leveled Compaction Strategy (LCS). This problem, which emerged during full table scans and affected processes like decommissioning, node addition, and repairs, was attributed to a low-level optimization in the 2.1 release that was missed in initial testing. Users employing LCS are urged to upgrade to 2.1.4 immediately, and those with relevant backups should contact support for restoration guidance. The release also fixes several other bugs, including issues with systemd in ScyllaDB AMI, RAID 0 data directory access, commit log error handling, and TLS connection errors. The developers are committed to improving the cluster test suite to prevent future oversights and will provide a detailed root cause analysis and subsequent enhancements.

The blog post covers a webinar discussing the integration of ScyllaDB with Kubernetes, addressing several key questions raised during the event. Helm and YAML manifest files are compared, with Helm being a packaging system that simplifies managing multiple manifest files through Helm Charts. It explains that if a ScyllaDB pod fails within a StatefulSet, it is replaced without changing its identity, and multi-datacenter deployments are supported by creating separate StatefulSets for each region. Pod readiness is determined by the status probe, specifically using nodetool status, while internal DNS management is handled by the StatefulSet. Despite a current performance degradation of 25% to 40% when using Docker containers, Kubernetes offers a unified deployment platform across various infrastructures, unlike AMIs that are AWS-specific. Upgrading ScyllaDB involves taking snapshots and reattaching upgraded containers to existing persistent storage. Helm Charts for ScyllaDB deployment are available and being integrated into main repositories, and the impact of data volume on deployment and decommissioning is discussed, highlighting increased throughput requirements. The post also promotes an upcoming webinar on powering Spark with ScyllaDB and provides links for further exploration of ScyllaDB, including downloads and user feedback.

ScyllaDB employs control theory to manage database compactions by automating resource allocation, thus maintaining system performance without user intervention. This approach addresses the challenges of balancing foreground requests and background processes, such as compactions, which are crucial for databases using Log Structured Merge trees. By leveraging control systems, ScyllaDB ensures that compaction bandwidth is dynamically adjusted to maintain a predictable system response, reducing the operational burden on users and adapting to workload changes autonomously. This method allows ScyllaDB to achieve equilibrium in resource usage, ensuring stable latency and throughput even as workloads fluctuate. The database's control system uses schedulers similar to industrial controllers, adjusting resource shares to manage the compaction backlog effectively, ultimately providing a resilient and efficient database environment.

Phillip Tribble's blog post, part of a series on the Mutant Monitoring System (MMS) and ScyllaDB training, explores the use of Apache Spark, Hive, and Superset to improve data analytics and visualization capabilities for the monitoring system amid increasing mutant attacks. Apache Spark is utilized for its distributed data processing capabilities, while Hive offers an SQL interface for querying data from the ScyllaDB cluster. Superset provides a graphical web interface to visualize the data, helping Division 3 to better analyze and respond to the threat. The post details the setup process for these tools, including running necessary containers and configuring communication between Spark, Hive, and ScyllaDB. By using these technologies, Division 3 aims to enhance their data-driven decision-making to better manage and monitor mutant threats.

Numberly, a programmatic advertising firm, transitioned from MongoDB to ScyllaDB to simplify operations and reduce costs while maintaining data consistency and performance. Initially, Numberly faced challenges with MongoDB and Apache Hive due to the operational burden of maintaining two copies of ID matching tables and the inefficiencies in handling read/write latencies required by their service level agreements. The firm explored Apache Cassandra but found it unsuitable due to concerns with Java-related performance issues. After rigorous testing, ScyllaDB was chosen for its ability to handle workloads with low latencies and operational simplicity, allowing Numberly to replace a 15-node MongoDB cluster with a more efficient 3-node ScyllaDB cluster. This transition resulted in significant cost savings, improved data consistency, and enhanced performance, with Numberly's Chief Technology Officer praising ScyllaDB for its robust and smart open-source community.