March 2018 Summaries

In the blog post "Exploring ScyllaDB on Kubernetes," Jesse Haber-Kucharsky discusses how Kubernetes, an open-source system for automating the deployment and management of containerized applications, can be utilized to deploy ScyllaDB for testing and integration purposes. The post outlines key Kubernetes concepts such as Pods, Services, and StatefulSets, explaining how they relate to deploying ScyllaDB. A practical example is provided, available on GitHub, which demonstrates setting up a ScyllaDB cluster on Kubernetes using Google Compute Engine. The example serves as a starting point for users to experiment with ScyllaDB on Kubernetes, encouraging further exploration into performance enhancements like CPU pinning and optimal storage configurations. The post emphasizes Kubernetes' ability to manage complex deployment requirements, such as resource scaling and distributed configuration management, and encourages users to build on the provided example and share their findings with the ScyllaDB community.

Phillip Tribble's blog post explores the implementation of multi-datacenter consistency levels in the ScyllaDB-powered Mutant Monitoring System (MMS), specifically focusing on the options of LOCAL_QUORUM and EACH_QUORUM. These consistency levels are crucial for determining how many replica nodes must acknowledge read or write operations for them to be considered successful. LOCAL_QUORUM ensures that a quorum of replicas in the local datacenter responds, offering low latency and a balance of consistency and performance, while EACH_QUORUM requires a quorum from all datacenters, providing higher data consistency at the expense of performance. The post includes practical guidance on setting up and managing these consistency levels using cqlsh and Docker, emphasizing the importance of data consistency across multiple datacenters and providing insights into the trade-offs between performance and resilience in a distributed database environment.

ScyllaDB 2.1.1, released by the ScyllaDB team, is a bugfix update for the ScyllaDB 2.1 stable branch, maintaining backward compatibility and supporting rolling upgrades. ScyllaDB, an open-source NoSQL database compatible with Apache Cassandra, is known for its superior performance and low latency. The new release addresses critical bugs, including a failure when querying partitions without clustering ranges and a regression in version 2.1.0 that caused crashes during initialization. It also resolves a gossip bug that led to segmentation faults when nodes were down. Users of ScyllaDB 1.7 or earlier versions are encouraged to upgrade to benefit from ongoing bug fixes and new features being developed for future iterations, such as ScyllaDB 2.2.

The blog post outlines the process of setting up a multi-datacenter configuration for the Mutant Monitoring System (MMS) using ScyllaDB, emphasizing the importance of disaster readiness and data protection in the event of a physical site failure. By expanding the ScyllaDB cluster to a second datacenter, the post demonstrates how to convert existing keyspaces to be stored in both datacenters, ensuring data is replicated and consistent across sites. The steps include initializing the ScyllaDB cluster, importing data, bringing up a second datacenter, and configuring keyspaces for multi-datacenter replication. The post also discusses the importance of consistency levels in maintaining data availability during site failures, emphasizing the use of LOCAL_ONE and LOCAL_QUORUM levels for ensuring data access when one datacenter is down. The tutorial provides practical guidance on using Docker commands to manage the datacenters and concludes by highlighting the enhanced data protection achieved through this setup.

KairosDB and ScyllaDB form a robust time-series solution by combining KairosDB's simple data ingestion and retrieval capabilities with ScyllaDB's large-scale, high-throughput backend. This integration, enhanced by KairosDB's native CQL support, allows for efficient data management, processing, and visualization, making it highly scalable and reliable. Performance tests demonstrate that ScyllaDB maintains low tail latencies and scales linearly with added KairosDB nodes, even when processing data from 180,000 sensors per second. The tests utilized AWS EC2 instances with specific configurations to handle large numbers of data points efficiently, showing that ScyllaDB's open-source version can manage substantial workloads without increasing resource allocation. The collaboration between the two databases ensures high availability and performance, making them an ideal choice for handling time-series data.

ScyllaDB Enterprise 2017.1.6 is a minor release focused on bug fixes for the 2017.1 branch, derived from the ScyllaDB open-source version 1.6, with additional bug fixes from subsequent versions and enterprise-specific updates. This release introduces a new feature that warns users when large partitions are compacted, with a configurable default threshold of 1000MB, and addresses an issue with commitlog segments remaining on disk after clean shutdowns, which is essential for future migrations to ScyllaDB Enterprise 2018.1. Customers are encouraged to upgrade to this version and can contact support for assistance, with additional information and download options available for various deployment methods, including AWS, local installation, and Docker.

Organizations are increasingly adopting Solid State Drives (SSDs) like Samsung's Z-SSD in data centers for enhanced performance and reduced latency, making them an ideal match for database solutions such as ScyllaDB. The Samsung Z-SSD, with storage capacities up to 800GB, boasts impressive performance metrics, supporting up to 3.2 GB/s in sequential read and write operations, and significantly outperforming other SSDs in latency tests. At the ScyllaDB Summit 2017, it was demonstrated that running ScyllaDB on the Z-SSD bridges the performance gap between memory and storage, achieving 44-23% of RAM performance at a reduced cost, with up to 75% of requests served from memory. This combination of ScyllaDB with the Z-SSD could lead to further improvements in memory/storage efficiency with future software updates, providing a cost-effective solution for handling large-scale data operations.

ScyllaDB participated in the Linux SCaLE 16x conference held at the Pasadena Convention Center, showcasing live demonstrations of its database running various workloads at booth #219, where attendees could engage in discussions about database projects and receive ScyllaDB merchandise. On March 11, 2018, ScyllaDB presented a session titled "Internal Architecture of a Modern NoSQL Datastore," which provided an in-depth examination of ScyllaDB's architecture, a C++ reimplementation of Apache Cassandra that offers unprecedented performance in the NoSQL sector. The presentation covered ScyllaDB's unique components, such as a userspace CPU Scheduler, a tailored memory allocator, and modifications to C++ exception handling, highlighting the benefits and trade-offs of moving certain functionalities from the Linux Kernel to userspace. Additionally, the session explored the role of Linux in supporting these innovations, the interfaces used in modern datastores, and the rationale behind these architectural decisions.

The blog post by Phillip Tribble explores the construction and functionality of a time series database using KairosDB on top of ScyllaDB, emphasizing its relevance for IoT devices and data analytics. KairosDB, a swift and open-source time series database, utilizes ScyllaDB as a storage backend, facilitating data ingestion through various protocols like Telnet, Rest, and Graphite, and offering a user-friendly web interface for query execution. The blog highlights the importance of designing an efficient database schema with a suitable primary key to optimize performance and avoid issues like data hotspots. It further underscores ScyllaDB's capability to handle data at scale with high performance and low latency, backed by benchmarks, and encourages readers to explore additional resources for a deeper understanding of the integration between KairosDB and ScyllaDB.

Part 5 of the Mutant Monitoring System series explores the use of Apache Zeppelin to visualize data from ScyllaDB, building on previous posts that addressed system recovery and cluster repairs. After re-establishing the system's operational status, the focus shifts to data analysis using Zeppelin, a Java-based platform compatible with various data sources such as MySQL and Spark, which allows users to run CQL queries and display results in tables and graphs. The blog post details the process of setting up ScyllaDB and Zeppelin using Docker, facilitating automatic data import and offering a step-by-step guide to creating and configuring a Zeppelin Notebook for data visualization. It highlights the visualization of various data points, including heat readings and mutant speed, providing insights into individual mutants like Jim Jeffries and Bob Loblaw, and concludes with a reminder of the ongoing efforts to enhance monitoring capabilities within the Mutant Monitoring System.

ScyllaDB's approach to improving Grafana dashboard management involves applying principles similar to CSS in HTML to separate repetitive styling from main logic, which simplifies modification and maintenance. By adding a 'class' attribute to JSON components and using a type definition file, the team reduced dashboard file sizes by about 30%, making it easier to copy panels and rows between dashboards. A Python program, make_dashboards.py, was developed to merge type JSON files and template files into a single file, allowing for dynamic inheritance and attribute overrides. This system enables streamlined management and could influence Grafana's future template mechanisms.