January 2018 Summaries

The blog post delves into the complexities of the Leveled Compaction Strategy (LCS) in ScyllaDB and Apache Cassandra, presenting it as a solution to the space amplification issue inherent in the Size-Tiered Compaction Strategy (STCS) but highlighting its introduction of a new challenge: write amplification. LCS, which organizes data into levels with exponentially increasing sizes, effectively reduces space amplification by minimizing temporary disk space usage and eliminating duplicate data storage. However, it exacerbates write amplification, leading to significantly higher disk write operations, which can severely impact performance in write-heavy or mixed read-write workloads. The post concludes by hinting at the next part of the series, which will explore the Hybrid Compaction Strategy, aiming to combine the benefits of LCS and STCS while mitigating their respective drawbacks.

In the continuation of the Mutant Monitoring System (MMS) blog series, the third installment explores the integration of Presto, a distributed SQL query engine, with ScyllaDB to analyze mutant data for potentially malicious activity. The system is used to track mutants' behavior by examining heat signatures, with a reading over 20 suggesting possible malicious intent. The blog demonstrates how Presto can be utilized to query data and assess the activities of specific mutants, such as Jim Jeffries and Bob Loblaw, by analyzing their heat levels. Jim's elevated readings potentially indicate malintent, while Bob's readings suggest otherwise. The integration of Presto allows for efficient querying and analysis of large datasets, enabling quick responses to suspicious activities. The narrative also hints at future developments, focusing on preparing for disaster scenarios and ensuring the system's robustness against node failures.

Phillip Tribble's blog post discusses strategies for reducing costs when running ScyllaDB, a NoSQL database, on public cloud providers like AWS, particularly in multi-data-center configurations. It highlights the potential for high data transfer costs when deploying ScyllaDB clusters in different geographical regions, such as the US and Asia, due to the charges associated with data sent over public networks. The post suggests using ScyllaDB's architecture, specifically the Ec2MultiRegionSnitch, which optimizes data routing by understanding network topology, to decrease unnecessary inter-region data transfers and thus reduce costs. Additionally, enabling inter-node compression is recommended as another method to cut down on network traffic expenses. These strategies are aimed at achieving significant savings without compromising the database's performance or resilience.

In the second part of the Mutant Monitoring System (MMS) blog series, the focus is on building a tracking system to monitor mutants by setting up a time-series database using ScyllaDB. This involves creating a keyspace and table designed to track various metrics such as name, timestamp, location, speed, velocity, heat, and telepathy powers of each mutant. The system employs a composite partition key for efficient data storage and retrieval, utilizing ScyllaDB's "DateTieredCompactionStrategy" to handle time-series data effectively. This strategy leverages time-based compaction to optimize data expiration and reduce the frequency of costly operations. By setting up this infrastructure, the MMS aims to help control the mutant threat by allowing for efficient data querying and management, paving the way for future analysis and system integration.

The blog post introduces the first of a four-part series on ScyllaDB's compaction strategies, focusing on the Space Amplification issue within the Size-Tiered Compaction Strategy (STCS), which is also applicable to Apache Cassandra. It explains that STCS, though the default strategy for both databases, suffers from space amplification, where disk space usage exceeds a perfectly compacted single sstable, leading to inefficiencies, particularly with SSDs. Experiments demonstrate that space amplification can lead to needing twice the disk space or more due to temporary duplication of data during compaction. The post highlights the severity of the problem when data is overwritten repeatedly and mentions that upcoming articles will explore other strategies like Leveled and Hybrid Compaction, which aim to address these issues while maintaining low write amplification.

ScyllaDB is now available on Oracle Cloud Marketplace, offering stable performance even in the event of a node failure, thanks to its Heat weighted load balancing feature introduced in ScyllaDB 2.0. Testing on Oracle Cloud Infrastructure's bare metal instances demonstrated ScyllaDB's ability to maintain throughput and latency performance, leveraging OCI's high CPU count, ample DRAM, and fast NVMe drives. During node drops, which are common in distributed systems, ScyllaDB can sustain approximately 2 million read operations per second by redistributing the load across remaining nodes without requiring additional tuning. Under normal conditions, with all nodes operational, the system can handle at least 4 million operations per second, storing data with a replication factor of three and maintaining quorum consistency levels.

Efficient data processing for applications can be achieved using streaming platforms like Apache Kafka and KSQL, moving away from traditional ETL methods that involve batch processing and numerous potential failure points. Apache Kafka serves as a high-throughput distributed streaming platform, while KSQL, developed by Confluent.io, enables real-time, scalable data processing using SQL-like syntax. This setup allows for continuous and interactive queries on data streams, such as monitoring logging attempts to identify suspicious activity. Integration with tools like Kafka Connect and ScyllaDB enhances data accessibility and storage, leveraging ScyllaDB's high performance and fault tolerance derived from Apache Cassandra. These technologies collectively support the development of modern applications with low latency and high efficiency, offering an alternative to the limitations of traditional data management systems.

In the wake of the Meltdown and Spectre vulnerabilities, software defenses against Meltdown, such as Kernel Page Tables Isolation (KPTI), have introduced significant performance penalties due to increased costs of system calls and context switches, affecting databases heavily. However, ScyllaDB, owing to its unique thread-per-core asynchronous architecture and efficient userspace I/O management, experiences minimal impact from these patches, with only a 6% slowdown observed in single-node benchmarks and a mere 2% in clustered environments, as opposed to the potential 30% performance hit seen in other complex server software. ScyllaDB's architecture allows it to handle tasks predominantly in userspace, minimizing context switches and system calls, which, coupled with its batching mechanism, reduces the overhead typically caused by system interactions in the wake of the Meltdown fix. This efficient architecture ensures that ScyllaDB remains resilient and performs well across various workloads despite the security patches.

Seastar is a systems framework that maximizes concurrency and parallelism in multi-core, multi-socket machines by emphasizing asynchronous programming and lock-free operations. Developed with inspiration from trends predicting increasing core counts over clock speed, Seastar employs a shared-nothing architecture that eliminates implicit memory sharing through thread-local data management, making it highly efficient and scalable. It allows programmers to focus on constructing correct program structures while the framework ensures efficient execution, comparable to actor frameworks like Akka and languages like Erlang. By offering granular control over IO and CPU scheduling and employing a composable future<> model, Seastar facilitates concurrent software development that can dynamically adjust parallel execution without altering the program structure. Its use in projects like the smf library illustrates its capability to deliver high-performance applications, such as the development of an RPC system with significantly lower latency compared to alternatives like gRPC and Apache Kafka.

In this detailed analysis, Alexander Sicular emphasizes the importance of understanding database architecture and capabilities beyond just high IOPS when ensuring Service Level Agreements (SLAs) during instability. The article contrasts Apache Cassandra with ScyllaDB, highlighting how ScyllaDB's architecture, which includes workload conditioning, heat weighted load balancing, and the use of the Seastar framework, provides resilience in the face of cluster instabilities without relying on the Java Virtual Machine (JVM). Through a case study of PagerDuty's 2017 service degradation, Sicular illustrates the challenges faced by distributed systems during node failures and the subsequent resource strains, underscoring the benefits of ScyllaDB's ability to dynamically manage resources and reduce latency spikes. The discussion extends to the importance of enterprise relationships with software vendors for optimal issue resolution, as exemplified by ScyllaDB's customer support. The piece concludes by appreciating transparency in root cause analyses, which offer valuable lessons in handling system failures and ensuring efficient recovery plans.