May 2019 Summaries

ScyllaDB Open Source 3.0.7 is a bugfix release within the ScyllaDB Open Source 3.0 stable branch, offering backward compatibility and support for rolling upgrades. This release addresses specific issues such as incorrect handling of empty counters in the "mc" SSTable file format, rare conditions causing missing rows during range scans, performance impacts due to a "use after free" scenario in the TimeWindowCompactionStrategy, and latency spikes caused by stalls when calculating the Heat Weighted Load Balancing (HWLB) for a large number of tables. Users are encouraged to report any problems encountered with this version.

Augury, a company specializing in predictive technology for industrial equipment, transitioned from MongoDB to ScyllaDB to address scaling and performance limitations as their dataset grew. This move was driven by the need for a more scalable data infrastructure that could handle high-volume analytics and provide real-time insights into machine conditions, crucial for their services, which reduce breakdowns and maintenance costs for customers. Augury's system leverages IoT sensors to collect machine data, using algorithms to deliver actionable insights and maintenance recommendations. ScyllaDB offered an architecture that supports both OLTP and OLAP use cases, reducing complexity and management overhead, and enabling efficient query processing with minimal configuration. The migration to ScyllaDB has been successful, providing Augury with a robust database solution that aligns with their operational needs and supports their expansion plans across North America and Europe.

ScyllaDB Cloud, a fully managed NoSQL Database as a Service (DBaaS) from ScyllaDB, operates on AWS I3 instances, leveraging ScyllaDB Enterprise's performance to support scalable real-time applications. It is API compatible with Apache Cassandra but is noted for delivering up to ten times the performance of other Cassandra implementations. In the May 2019 update, ScyllaDB Cloud introduced several new features, including Two Factor Authentication (2FA), which requires the Google Authenticator app, and the availability of Multi-Data Center Clusters, which users can enable by contacting ScyllaDB support. Additionally, the platform now runs on ScyllaDB Enterprise 2019.1.0, with automatic upgrades as new releases are made available.

AWS has introduced the I3en family of instances, designed to provide an optimal solution for storage-intensive workloads by offering up to 60TB of fast NVMe storage, thereby addressing the limitations faced by the I3 family in terms of disk-to-CPU ratio. This new family features Xeon Platinum 8175M processors, which are faster and have more cores than their I3 counterparts, and utilizes the Nitro KVM-based hypervisor, enhancing performance for I/O-driven tasks. The I3en.24xlarge instance, the largest in the family, showcases impressive performance metrics, including 100Gbps networking and high IOPS rates, making it suitable for demanding applications such as ScyllaDB, which can effectively utilize these resources to handle a 45TB dataset with millions of data point ingestions per second and low latency reads. This advancement offers AWS users greater flexibility and cost-effectiveness when selecting instances based on storage needs, providing a lower price per terabyte of data while maintaining competitive processing capabilities.

ScyllaDB's workload prioritization mechanism, introduced in ScyllaDB Enterprise Release 2019.1, aims to efficiently manage both OLTP and OLAP workloads on the same database cluster by allocating system resources based on user-defined shares. This feature allows for co-hosting of different workloads, addressing the traditional inefficiencies and costs associated with segregated clusters for OLTP and OLAP processes. By using ScyllaDB's internal schedulers and resource management tools, workload prioritization ensures that each process receives an appropriate share of resources, thus maintaining a balance even under high system loads. Test results demonstrate that with workload prioritization enabled, OLTP workloads experience minimal performance degradation when OLAP tasks are executed concurrently, with a controlled impact on latency and throughput. This mechanism provides a flexible and cost-effective solution to optimize resource utilization and meet service level agreements while avoiding the duplication of hardware setups typically required for handling conflicting workloads separately.

Yahoo! Japan, a dominant internet brand in Japan since 1996, utilizes a diverse technological infrastructure to support its wide array of services, including mobile advertising, online stores, and payment systems. To manage the data demands of its extensive user base, the company employs a range of databases, including Oracle, MySQL, Teradata, HBase, and notably, Apache Cassandra and ScyllaDB for NoSQL needs. At the ScyllaDB Summit 2018, Yahoo! Japan's engineers discussed their testing of ScyllaDB against Apache Cassandra, highlighting several performance issues with Cassandra related to Java Virtual Machine's garbage collection and node maintenance. Their benchmarks demonstrated that while Cassandra and ScyllaDB performed similarly at lower thread counts, ScyllaDB significantly outperformed Cassandra as thread counts increased, offering better latency and resource utilization, despite some operational challenges. This testing underscored ScyllaDB's potential advantages in performance and stability over Cassandra, suggesting that ScyllaDB could handle higher loads more efficiently without frequent crashes, although Cassandra's many tunable parameters offer both flexibility and complexity.

ScyllaDB Manager 1.4 has been released, introducing various enhancements for both ScyllaDB Enterprise and Open-Source users, notably in the areas of repair processes and SSH connectivity configuration. The update simplifies the SSH setup via a new script and integrates a validation step to ensure connectivity. A dry-run mode for the repair command enables users to inspect evaluated patterns without scheduling repairs, and improvements have been made for multi-datacenter cluster repairs, including better handling of replication strategies. The progress reporting now offers more detailed feedback, including error causation and task arguments, and introduces new filtering capabilities. Additionally, the status command has been updated to include REST API connectivity checks, and improved error reporting now provides clearer feedback within the interface.

In the wake of the Meltdown and Spectre vulnerabilities, the emergence of new processor vulnerabilities like ZombieLoad has highlighted the ongoing risks of side-channel attacks, particularly those exploiting Intel's Hyperthreading technology. These vulnerabilities have prompted recommendations to disable Hyperthreading and apply patches, but long-term protection demands strategic architectural and operational decisions. To mitigate these risks, cloud providers and infrastructure engineers are advised to minimize shared infrastructure and avoid multi-tenancy, as these practices can expose systems to side-channel attacks. ScyllaDB is presented as a robust solution, capable of both horizontal and vertical scaling, ensuring efficient resource utilization and enhanced security by reducing the attack surface. By adopting fewer, larger nodes, organizations can better defend against such vulnerabilities, marrying security with economic feasibility and reducing the potential impact of future side-channel exploits.

Ryan Stauffer's blog post discusses the integration of ScyllaDB and JanusGraph to create a graph data system that addresses complex data management needs, particularly in the context of a Master Data Management project with a cybersecurity company. The system aims to provide a comprehensive view of customer and supply chain activities by merging disparate data sources into a unified model, leveraging the flexibility and performance of graph databases. The post highlights the benefits of using a graph data system, such as the ability to evolve data models over time, enforce flexible schemas, and support both OLTP and OLAP workloads using the Gremlin graph traversal language. The deployment process involves setting up ScyllaDB and Elasticsearch on Google Cloud Platform, using Kubernetes for scalability and repeatability, and implementing JanusGraph as a server to facilitate client requests. The article provides detailed instructions on deploying the system, defining an initial schema, and loading data, serving as a guide for readers interested in building similar systems.

SteelHouse, an Ad Tech platform, successfully transitioned from using Apache Cassandra to ScyllaDB to improve the performance and reliability of their globally distributed ad-serving system. The switch was driven by persistent timeout issues with Cassandra, which was unable to maintain the required service level agreements (SLAs) for latency-sensitive operations. ScyllaDB offered self-tuning capabilities, better performance, and reduced hardware needs, which simplified management and increased efficiency. The transition, though initially intense, resulted in significant improvements, including the elimination of timeouts, enhanced responsiveness, and smoother peak season operations during Black Friday/Cyber Monday, contributing to a record-breaking holiday season in terms of both technical stability and business performance. The migration saved the team substantial maintenance time, improving productivity by approximately 20%, and solidified confidence in ScyllaDB's ability to handle billions of monthly requests effectively.

ScyllaDB's release of Open Source 3.0.6, announced by Tzach Livyatan on May 7, 2019, is a bugfix update for the ScyllaDB Open Source 3.0 stable branch that maintains backward compatibility and supports rolling upgrades. This version addresses several issues, including incorrect IP addresses being printed during CQL batch operations, malfunctioning secondary indexes for clustering keys during updates, potential doubling of disk space usage during nodetool cleanup, and various schema change problems such as race conditions and indefinite delays in schema change statements. Additionally, it fixes a memory allocation error and an issue where resharding could cause ScyllaDB to exit unexpectedly. The release is available through Docker, binary packages, and EC2 AMI, and users are encouraged to report any problems encountered.

ScyllaDB Enterprise 2019.1 is a high-performance NoSQL database designed for real-time big data workloads, boasting a significant increase in throughput and reduced latency compared to Apache Cassandra. Built in C++ for a close-to-the-hardware design, it includes innovations such as Workload Prioritization, which allows concurrent OLTP and OLAP workloads within a single cluster without compromising performance. This release integrates features from ScyllaDB Open Source 3.0, such as Materialized Views and Global Secondary Indexes, enhancing querying efficiency and storage management. ScyllaDB Enterprise 2019.1 introduces improvements in streaming, cache eviction, and a new storage format that reduces data footprint, making it compatible with Apache Cassandra 3.x. Additionally, it offers enhanced role-based access control, support for JSON, and datetime functions, while introducing a new CPU scheduler and compaction controllers to optimize internal workload management. These advancements position ScyllaDB Enterprise as a robust solution for scalable and reliable data management, available for download or a 30-day trial for new users.

The article provides a detailed comparison between ScyllaDB Cloud and Google Cloud Bigtable, two NoSQL Database-as-a-Service (DBaaS) offerings, focusing on performance, cost, and scalability under various workload scenarios. ScyllaDB Cloud emerges as the more cost-effective solution, being 1/5th the cost of Cloud Bigtable under optimal conditions and performing 26 times better with real-world, unoptimized data distributions. The benchmark study highlights ScyllaDB's ability to maintain its service level agreements (SLAs) more efficiently than Cloud Bigtable, which struggles to meet the required 90,000 operations per second without significant scaling and cost increases. Under a Zipfian distribution, which simulates real-world conditions with uneven data access patterns, ScyllaDB Cloud continues to outperform, handling 90,000 requests per second with significantly lower latency compared to Cloud Bigtable's 3,450 requests per second. The study also explores the performance of both databases when accessing a single hot row, finding that ScyllaDB Cloud manages nearly 200 times the throughput of Cloud Bigtable. The article emphasizes the importance of optimizing data models to improve performance and suggests that ScyllaDB Cloud's flexible deployment options offer greater user autonomy, without locking them into a specific cloud provider.