March 2019 Summaries

ScyllaDB Monitoring Stack version 2.2 introduces a new CQL Optimization Dashboard designed to help users identify and resolve potential issues with queries, data models, and drivers to improve performance. The dashboard features gauges and graphs that highlight various inefficiencies such as non-prepared queries, non-token-aware drivers, non-paged queries, reversed CQL reads, and ALLOW FILTERING queries, all of which can affect the system's performance. Users are encouraged to keep these metrics low, ideally close to zero, to ensure optimal performance. The dashboard also addresses cross-data-center read requests that can add unnecessary overhead, emphasizing the importance of monitoring these metrics continuously during application development and operation. Through this tool, ScyllaDB aims to provide users with actionable insights to enhance their database operations and performance.

Glauber Costa's blog post provides a comprehensive guide on best practices for optimizing ScyllaDB applications, particularly for those transitioning from Apache Cassandra or new to the ScyllaDB environment. It emphasizes the importance of monitoring with the ScyllaDB Monitoring Stack, planning key structures for effective data distribution, and designing tables with queries in mind. The article also advises on managing partition sizes, using batching judiciously, and keeping data inserts and collection sizes manageable to maintain database performance. It highlights the benefits of prepared statements to minimize execution costs and the necessity of maintaining high parallelism to maximize ScyllaDB's capabilities. Additionally, it discusses the significance of choosing the right load-balancing policies to ensure efficient data access and avoid latency issues, offering practical tips for both new and experienced users to enhance their ScyllaDB setups.

ScyllaDB Open Source 2.3.4 is a bugfix release that addresses a critical race condition between schema updates and compaction processes, which could cause data to be written to incorrect columns, potentially leading to crashes or errors. The release is backward compatible, supports rolling upgrades, and is part of the 2.3 stable branch, although users are encouraged to upgrade to the latest stable version, ScyllaDB 3.0. Additional issues resolved include crashes related to multi-column IN restrictions and connection imbalances caused by shard-aware drivers. The release is available through Docker and binary packages, and users are advised to report any encountered problems.

Discord's rapid growth in user base brought challenges in scalability and uptime, prompting the exploration of efficient database solutions like ScyllaDB to manage the vast volume of messages exchanged daily. Mark Smith, Director of Engineering at Discord, highlighted the importance of opinionated systems, drawing from past experiences with Apache Kafka, emphasizing that while open-source technologies are easy to start, scaling them efficiently requires significant expertise. ScyllaDB's built-in configurations and monitoring tools provide a streamlined approach to deployment, reducing the complexity and decision-making burden associated with managing large-scale systems. Despite experiencing a production incident due to capacity issues, Discord quickly resolved it by expanding the cluster, showcasing ScyllaDB's ability to simplify troubleshooting and enhance operational reliability. Mark advocated for the value of having strong opinions in system design, suggesting that limiting configuration options can reduce mental overhead and improve efficiency, a principle that has proven beneficial for Discord's operations.

ScyllaDB Monitoring Stack 2.2, an open-source monitoring solution based on Prometheus and Grafana, has been released to support ScyllaDB Enterprise and Open Source versions. This release introduces a new CQL optimization dashboard to aid in identifying development issues such as non-prepared statements and remote data center requests, as well as unified target files to simplify Prometheus configuration. Enhancements include a per-machine dashboard for enterprise users, the ability for the Prometheus container to use the current user ID and group, and improvements to the shutdown process with the kill-all.sh script. Additional updates feature the removal of version numbers from dashboard names, an update to Alertmanager 0.16, and the necessity of aligning Data Center names for the new optimization dashboard. These changes aim to streamline monitoring tasks and improve the efficiency and user experience of ScyllaDB deployments.

ScyllaDB Enterprise 2018.1.11 is a minor release aimed at resolving specific bugs within the 2018.1 branch of the database, with a significant fix addressing a rare race condition that can occur between schema updates and initiated compactions, potentially causing errors or crashes. The update also tackles issues such as unbalanced connections due to shard-aware drivers, potential long reactor stalls during node rebuilding in large clusters, and the failure of bootstrapping nodes due to premature ring joining without schema application. The release encourages customers to upgrade in collaboration with the ScyllaDB support team and offers guidance on upgrading from previous versions, enhancing overall database stability and performance.

ScyllaDB Open Source 3.0.4, a bugfix release from the ScyllaDB 3.0 stable branch, addresses several issues to enhance performance and stability while maintaining backward compatibility and supporting rolling upgrades. The release primarily resolves a rare race condition between schema updates and compaction processes that could lead to data being written to incorrect columns, potentially causing crashes or errors. Additionally, fixes include resolving commit log overlapping issues, improving back pressure management in materialized views, balancing shard connections in ScyllaDB drivers, and mitigating large allocation requirements for MC file formats. Other improvements address a variety of technical issues, such as unaligned DMA addresses, JSON insertion limitations, authentication memory access, and streaming and bootstrap challenges, while also establishing TLSv1.2 as the default for CQLSh.

GPS Insight, a company that tracks over 140,000 vehicles, transitioned from using an SQL-based SAP Adaptive Server Enterprise (ASE) to a NoSQL solution for managing its real-time vehicle data due to scalability and performance challenges with their existing system. Initially considering Cassandra, they opted for ScyllaDB after a successful proof of concept demonstrated superior loading speed and scalability under increased loads. The implementation involved a hybrid deployment of on-premises and cloud-based nodes and necessitated adjustments to business logic and data management strategies, including the adoption of a diagnostic Data Definition Language (DDL) to facilitate communication between ScyllaDB and their SQL system. This transition not only improved data management efficiency and performance but also allowed the SQL team to upgrade their systems without the burden of massive data loads, showcasing ScyllaDB as an effective solution for handling large-scale IoT data in the fleet tracking industry.

The ScyllaDB Spark Migrator was developed with several design goals, including high resource efficiency and ease of use, to facilitate the migration of data from Cassandra to ScyllaDB using Apache Spark. The Migrator minimizes data shuffles to maintain performance and allows for minimal tuning by default, with parallel data transfer configurations and timestamp preservation capabilities. The process involves reading the schema from Cassandra, customizing queries to include TTL and WRITETIME timestamps for non-key columns, and using a combination of DataFrames and RDDs to manage schema modifications and data transformations. Additionally, the Migrator tracks the progress of data transfer by monitoring token ranges and using accumulators to save progress, enabling resumable migrations. The tool relies on a modified version of the Cassandra connector to achieve these functionalities, providing a robust solution for scalable and efficient NoSQL data migration.

Tantan, a locality-based dating app with over 10 million daily active users, connects singles by offering profiles that incorporate photos, likes, and interests, allowing users to match through swipes and interact via texts and multimedia. To manage real-time user location data and maintain low latency for its mixed read and write workloads, Tantan initially used two 16-node Cassandra clusters but faced significant latency issues, largely due to garbage collection. By transitioning to an 8-node ScyllaDB cluster, Tantan effectively addressed these challenges, reducing latency and database management costs, while benefiting from ScyllaDB's efficient handling of real-time data and support from its community.