June 2019 Summaries

ScyllaDB held an internal Developer Summit in Herzliya, Israel, bringing together its global team from 14 countries for a week of collaboration, learning, and team-building activities. The event featured intensive sessions, including presentations and hackathons, focused on ScyllaDB's core projects and future developments like user-visible features and testing processes. The summit also emphasized ScyllaDB's corporate culture of fairness, innovation, and accountability, providing opportunities for developers to bond through activities such as a day trip to Masada, a beach party, and team nights in Tel Aviv. The company highlighted open positions available for developers, encouraging talented individuals to join their diverse team.

ScyllaDB Monitoring Stack version 2.4 introduces enhancements designed to improve the monitoring experience for ScyllaDB Open Source 3.1, including refined dashboard views and new features for error tracking and node management. The update allows users to analyze shards at a granular level and offers the ability to choose aggregation functions, aiding in the identification of specific issues within instances. A new error dashboard has been added to assist engineers in diagnosing system malfunctions, while clear labeling distinguishes between metrics originating from coordinators and replicas, important for understanding data replication and consistency levels. Additionally, an overview dashboard now includes a table detailing node information such as datacenter affiliation, IP address, version, and status, with status metrics available only in ScyllaDB 3.1 and later versions. Users are encouraged to download the open-source update from GitHub and provide feedback through private contact or community platforms like Slack.

ScyllaDB Manager 1.4.1 is a maintenance release aimed at fixing bugs in the ScyllaDB Manager 1.4, a system designed to automate maintenance tasks for ScyllaDB clusters. Key improvements in this release include restoring the fallback logic to node IP for cases where remote dial fails, which is particularly relevant for users upgrading from version 1.3 and facing connectivity issues. Additionally, the update addresses the issue of the manager hanging by checking for the existence of a keyspace before attempting to create it, which benefits users with limited CQL privileges. This release caters to both ScyllaDB Enterprise customers and open-source users with a node limit, and it improves the overall reliability and functionality of the ScyllaDB Manager.

The blog post discusses the challenges and considerations involved in optimizing ScyllaDB cluster sizing to avoid node sprawl and overprovisioning, which have become common issues in modern data centers. It emphasizes the importance of leveraging ScyllaDB's ability to utilize modern hardware efficiently, thereby reducing costs and operational overhead. Key factors in cluster planning include storage, CPU, memory, network interfaces, and data growth projections, with a focus on achieving high utilization and low latency. The post offers guidance on selecting the appropriate hardware and configuration, whether deploying on-premises or in cloud environments such as AWS and Google Cloud. It also highlights the significance of monitoring, using ScyllaDB's Monitoring Stack to ensure optimal performance and resource allocation, and discusses the benefits of deploying across multiple availability zones or regions for enhanced resilience and reduced latency.

Opera, a web browser company founded in 1995, transitioned from using Cassandra to ScyllaDB for its Opera Sync service to address persistent technical issues, including high loads and GC-related problems. This decision was influenced by ScyllaDB’s demonstrated efficiency and compatibility with Cassandra, as observed by Opera's Project Technical Manager, Rafał Furmański, during a Cassandra Summit. The migration to ScyllaDB involved a phased approach, allowing Opera to move users gradually and ensure system stability. The switch has resulted in significant performance improvements, with ScyllaDB operating three times better in terms of operations per second and latency under test conditions. Post-migration, Opera reports a reduction in hardware requirements, improved load management, and a decrease in the operational strain on their engineering team, enabling them to focus on developing new features rather than on database maintenance.

ScyllaDB's documentation plays a crucial role in the product's adoption by facilitating knowledge transfer to its community, driven by the contributions of technical writers such as Laura Novich, who combines her technical and educational background to improve user understanding. The documentation team follows an open-source philosophy, focusing on timely updates and incorporating user feedback to refine the content. Recent enhancements to ScyllaDB documentation include improved search capabilities, a glossary of terms, and detailed pages on new features like Workload Prioritization and Role-Based Access Control. The documentation covers various aspects, including tools, drivers, and best practices for running ScyllaDB, and emphasizes not only the "how" but also the "why" and "how to do better" of using ScyllaDB. Users are encouraged to participate in the documentation's improvement through GitHub and Slack, ensuring it remains a valuable resource for developers and administrators.

Hang Chan of Dstillery discusses their transition from Apache Cassandra to ScyllaDB for managing their large-scale data needs, which involve processing device-level data to support custom audience solutions in the marketing and advertising industry. Dstillery faced challenges with Cassandra, including a persistent failure rate that could not be mitigated by adding more nodes, leading them to adopt ScyllaDB for its near-zero failure rates and self-optimizing capabilities. By migrating to ScyllaDB, Dstillery managed to significantly reduce failure rates and server count, while benefiting from simplified configuration that automatically adjusts to workload demands. ScyllaDB is integral to Dstillery's infrastructure, providing rapid data lookup capabilities and supporting applications like real-time bidding and geodata authenticity verification. The company continues to use a combination of databases and tools, including MySQL, memcached, Hadoop/MapReduce, Apache Kafka, and their in-house middleware, Mercuryd, for a robust big data adtech architecture.

ScyllaDB University has introduced a new, comprehensive module on data modeling as part of its ScyllaDB Essentials course, aimed at enhancing users' understanding of key concepts such as primary, partitioning, and clustering keys, which are crucial for effective database management. Unlike traditional relational database modeling that prioritizes domain and entity relationships, ScyllaDB and Cassandra require early consideration of application-specific queries, integrating both domain knowledge and application requirements. Since its launch at the ScyllaDB Summit 2018, the University has successfully engaged hundreds of participants and continues to expand its offerings, including the Mutant Monitoring System course, which uses a narrative approach to cover topics from basic cluster setup to advanced data replication across multiple datacenters. The platform's courses are self-paced, free, and designed to complement on-site training, with plans for future interactive improvements to enhance learning experiences.

Tomer Sandler's blog post provides a detailed guide to migrating data to ScyllaDB Cloud, a managed NoSQL Database as a Service (DBaaS), with a focus on databases compatible with SSTable formats like ScyllaDB Open Source, Cassandra, or DataStax Enterprise. It outlines the step-by-step process of migration, beginning with assessing cluster requirements, such as throughput, latency, and data size, and then proceeds to snapshotting the source cluster's data, uploading it to an S3 bucket, and preparing it for ScyllaDB Cloud. The deployment of ScyllaDB Cloud clusters can be tailored to specific needs, including single or multi-region setups and varying replication factors. Data migration is performed using tools like the sstableloader or the ScyllaDB Spark Migrator, which utilizes Apache Spark for efficient data transfer. The post also emphasizes the importance of pointing clients to the new database and ensuring everything functions smoothly, offering support for troubleshooting any issues that may arise during the migration process.