September 2019 Summaries

The blog post discusses the use of systemd slices as an efficient method to achieve resource isolation for complex database servers, specifically focusing on ScyllaDB. Unlike Docker, which can be overkill with its OS image requirements and network abstraction, systemd slices offer a lighter alternative by utilizing Linux cgroups to manage resource allocation among processes. This method allows ScyllaDB to allocate resources effectively between its main server and auxiliary applications, such as backup tools and metrics agents, by defining hierarchical slices with specific resource priorities. The post highlights a benchmark test demonstrating the benefits of systemd slices, where ScyllaDB maintains high throughput and low latency even under CPU stress, thanks to prioritized resource allocation. The approach emphasizes the ability to manage resources without the overhead of container technologies, providing a native solution for resource isolation that can extend to ad hoc commands and various resource types like CPU, memory, and I/O.

ScyllaDB has made significant progress over the past 18 months, fueled by a recent round of funding that has enabled the expansion of its development team and advancements in its database ecosystem. The company has increased its number of paying customers fivefold and introduced a range of new features, including Workload Prioritization, which allows for efficient resource management and isolation of workloads. ScyllaDB has also enhanced its offerings by addressing common challenges in NoSQL databases, such as compaction and repair, while supporting additional features like materialized views and scalable indexes. The release of ScyllaDB Cloud offers a fully managed service on AWS, and the company plans to expand its cloud support to Azure and GCP. ScyllaDB is seeing increased interest from companies seeking low-latency, high-availability solutions, including those transitioning from other databases like Cassandra and MySQL. The upcoming ScyllaDB Summit will showcase new features such as Lightweight Transactions and Change Data Capture, further positioning ScyllaDB as a versatile infrastructure solution for data-driven industries.

MediaMath, an enterprise-grade ad tech platform provider, leverages ScyllaDB to enhance its global operations in creating an accountable and transparent supply chain for advertisers. Initially using Apache Cassandra, MediaMath faced challenges with operational complexity and maintenance, prompting their switch to ScyllaDB, which offered compatibility with Cassandra along with improved performance. This transition allowed MediaMath to reduce operational overhead, achieve higher data throughput, and maintain read latency below the 10-millisecond threshold, ultimately leading to 99.9999% availability. By unifying demand-side and data management platforms, MediaMath facilitates programmatic buying strategies, optimizing campaigns across various media touchpoints, while ScyllaDB enhances data retention capabilities, enabling MediaMath to deliver increased value to its clients without additional costs.

ScyllaDB Monitoring Stack 3.0.1 has been released, providing an open-source monitoring solution for ScyllaDB Enterprise and ScyllaDB Open Source, utilizing Prometheus and Grafana for comprehensive data visualization and analysis. This update primarily introduces support for ScyllaDB's new Project Alternator, an Amazon DynamoDB compatible API, featuring a dedicated Alternator dashboard. Users can access the new dashboard by upgrading to version 3.0.1 and running the start_all.sh script with the -v master option, as Alternator is not yet part of a formal ScyllaDB release. The update does not include additional fixes or enhancements beyond this dashboard integration.

ScyllaDB has introduced Project Alternator, an open-source tool that ensures API-level compatibility with Amazon DynamoDB, allowing users to migrate their data seamlessly to ScyllaDB without altering client code. This initiative expands the capabilities of the ScyllaDB Migrator, originally designed for Apache Cassandra, by enabling data transfer from DynamoDB to ScyllaDB using Apache Spark-based parallel processing. The migration process involves configuring source and target settings, submitting the Spark application, and adjusting throughput variables to suit provisioned resources, which can be monitored via the Spark web UI. The successful data migration is verified by matching key counts and conducting random key queries across both databases. Future updates aim to incorporate DynamoDB's stream interfaces to facilitate live migrations without downtime, and a webinar is scheduled to provide more insights into Project Alternator.

ScyllaDB's Alternator project aims to provide an open-source, DynamoDB-compatible API that allows applications developed for Amazon DynamoDB to run seamlessly on ScyllaDB. As ScyllaDB evolved from its beginnings as a re-implementation of Apache Cassandra, it now offers significant performance and cost advantages, such as the ability to handle millions of operations per second with low latency on fewer resources, resulting in substantial cost savings. The Alternator project is not yet generally available but is incorporated into the ScyllaDB source code, with plans for further development and a future enterprise release. ScyllaDB emphasizes openness, offering a fully open-source platform under the AGPL license, allowing users to run it in various environments without vendor lock-in. The project seeks to enhance compatibility and performance with features such as workload prioritization, light-weight transactions, and future additions like user-defined functions and change data capture. ScyllaDB continues to disrupt traditional commercial vendors by promoting an open and collaborative development model.

In high throughput, read-heavy systems like ScyllaDB, effective caching is crucial for performance, as it significantly speeds up data retrieval compared to accessing storage. The article explores the scenarios and challenges of caching time-series data, particularly in IoT applications, by examining how ScyllaDB handles queries for time ranges. It highlights the differences between querying with a fixed time range versus using an open-ended range, explaining that while both methods appear similar, they have distinct impacts on the cache's efficiency. Using an open-ended range can optimize performance by allowing the cache to stay updated with new data, thus serving future queries entirely from the cache without requiring storage access. The article underscores the importance of understanding cache operations to avoid anti-patterns and ensure optimal performance in systems reliant on time-series data.