February 2020 Summaries

ScyllaDB has released version 3.2.2 of its open-source database, a maintenance update for the 3.2 stable branch, focusing on bug fixes and backward compatibility with previous 3.x.y versions. The release addresses several critical issues, including a bug with Hinted Handoff and Counters that could lead to incorrect counter values, problems with materialized view updates involving future partition tombstones, and a negative varint value encoding issue. Security and stability improvements have also been made, such as fixing a potential YAML reading issue, preventing crashes related to invalid time UUIDs and write operations, and enhancing the handling of Lightweight Transactions (LWT). Users are encouraged to upgrade and report any further issues, with various installation options available, including Docker, binary packages, and EC2 AMI.

ScyllaDB Enterprise 2019.1.5 has been released as a production-ready patch focused on enhancing the stability and robustness of its high-performance NoSQL database for real-time big data workloads. Key improvements aim to reduce memory usage and fix various bugs, particularly those affecting users utilizing both Hinted Handoff and Counters. The release addresses issues related to data correctness, CQL error formatting, and system stability, including problems with nodetool operations, UUID handling, and memory allocation in complex queries. It also features a new configuration option to manage shard-aware drivers and addresses problems with Java installations on specific Ubuntu versions. Customers are encouraged to upgrade in coordination with the ScyllaDB support team to benefit from these enhancements.

The blog post discusses the use of the GoCQLX package, an extension of the GoCQL driver, for interacting with a ScyllaDB cluster, enhancing developer productivity without compromising performance. Inspired by sqlx, GoCQLX offers additional features such as query builders, named parameters, and support for binding parameters from various data structures, as well as functions for loading query results into structs. The post describes a sample application that connects to a ScyllaDB cluster, performs basic database operations like displaying, inserting, and deleting data from a table, and outlines the implementation of these operations using GoCQLX's convenient functions. It emphasizes the modular and unified access to queries through struct constructs, and the main function demonstrates connecting to the database and executing these queries. For further details, including code and execution instructions, readers are encouraged to explore the full lesson at ScyllaDB University.

Rohit Saboo's blog post discusses the challenges and solutions in building a consistent trip processing system at Nauto, a real-time AI-powered driver behavior platform. The focus is on achieving consistency in a distributed system while balancing the trade-offs between consistency, availability, and partition tolerance, as dictated by the CAP theorem. Nauto's "trip builder" processes and analyzes large volumes of data using a distributed time-series merging solution based on a new ACID paradigm, which provides robust application-level guarantees. ScyllaDB was chosen as the database due to its reliability, cost-effectiveness, and suitability for high-throughput operations. The solution involves a coordination-free algorithm that merges trip segments using a monotonic function ensuring data consistency, reliability, and low operational overhead. By leveraging ScyllaDB's capabilities, Nauto has successfully optimized its system for high availability and scalability, enabling it to predict and prevent high-risk driving events, thus avoiding over 25,000 collisions and saving nearly $100 million for its commercial fleet clients.

ScyllaDB Open Source 3.1.3 is a bugfix release for the 3.1 stable branch of ScyllaDB, ensuring backward compatibility and support for rolling upgrades. This release addresses several stability and performance issues, such as excessive IO resource use during updates of materialized views, failures with User Defined Types (UDTs), and read amplification caused by materialized views. It is essential to note that ScyllaDB 3.2.1 is the latest stable release, and version 3.3 is in the release candidate phase, with support for the 3.1.x release ending once 3.3 is officially released. The update also provides specific instructions for users upgrading from ScyllaDB 3.1.0, emphasizing the need to enable compatibility mode in the scylla.yaml file.

Apache Kafka, a cornerstone of modern Big Data architectures, has gained widespread adoption since its inception in 2011, enabling real-time stream processing across a multitude of companies, including many Fortune 100 firms. The introduction of a new shard-aware Kafka sink connector for ScyllaDB, which is API-compatible with Apache Cassandra, aims to enhance performance by leveraging ScyllaDB’s shard-per-core, shared-nothing architecture. This development is particularly significant for users who previously utilized the Kafka Cassandra Connector but found it insufficient in exploiting ScyllaDB's full capabilities. The integration of ScyllaDB's high-performance NoSQL database with Apache Kafka promises scalable, high-throughput solutions for modern event streaming data architectures, offering flexibility through various combinations of open-source, enterprise-grade, and cloud-hosted solutions. This innovation is seen as the final component needed to build the next generation of Big Data applications, with additional insights available through webinars and case studies focused on real-time streaming and IoT deployments.

ScyllaDB, a highly scalable and fast NoSQL database, has been lauded by various organizations for its performance and ease of use, as highlighted in testimonials recorded at the ScyllaDB Summit. Companies ranging from large corporations like Comcast to start-ups have shared their positive experiences, citing significant improvements in latency, reduced administrative workloads, and enhanced system reliability. Users have emphasized the seamless migration from databases like Apache Cassandra, with minimal changes required. For instance, Comcast reported a 95% reduction in latency, while Tubi achieved a decrease from 300 ms to 10 ms. Notable mentions include Opera's ability to synchronize experiences across platforms and FireEye's enhanced performance for back-end systems. The fully managed ScyllaDB Cloud was praised for its low maintenance, enabling companies like Mistaway Systems to focus more on their core applications. The ScyllaDB Summit further showcased detailed customer use cases and technical presentations, reinforcing the diverse benefits of adopting ScyllaDB in different sectors.

The ScyllaDB Summit 2019 Training Day featured both novice and advanced tracks, offering hands-on exercises and quizzes to enhance participants' skills in ScyllaDB and NoSQL technologies. The novice track covered foundational topics such as the CAP theorem, data modeling, and migration techniques, with emphasis on ScyllaDB's design goals of high availability, scalability, and performance. Participants learned about key concepts including nodes, keyspace, and consistency levels, as well as best practices for data migration using tools like Kafka and the ScyllaDB Spark Migrator. The advanced track delved into complex topics like advanced data modeling, materialized views, secondary indexes, and various compaction strategies, including the novel Incremental Compaction Strategy. The event also highlighted ScyllaDB Manager for automated cluster operations, advanced monitoring techniques, and the ScyllaDB Operator for Kubernetes integration. Attendees had access to related content and videos through ScyllaDB University, earning certificates for completed courses, while ongoing discussions and learning opportunities were encouraged via Slack channels.

ScyllaDB was honored with an InfoWorld 2020 Technology of the Year Award, recognized alongside prominent technologies such as Tableau and Datadog, reflecting its significant advancements and contributions to modern infrastructure. This enterprise database product, developed to achieve feature parity with Apache Cassandra, has begun to challenge DynamoDB with its new compatible API. ScyllaDB, implemented in C++, focuses on performance optimization by prioritizing real-time transactions and efficiently utilizing CPU resources even at 100% capacity. The database offers durable and fast Lightweight Transactions, global and local secondary indexes, and an elegantly implemented Change Data Capture feature. It also supports large data objects and nodes, requires no manual tuning, and incorporates io_uring for improved performance. ScyllaDB integrates with various tools like Confluent’s Kafka and Datadog for enhanced functionality, and its success is attributed to user-driven development and continuous improvements, with a promise of future initiatives.

FireEye, now known as Trellix, is a NASDAQ-listed cybersecurity firm that provides a comprehensive range of security solutions, including threat intelligence platforms and managed defenses, catering to clients across various sectors such as telecommunications, healthcare, finance, and government. At the ScyllaDB Summit 2019, representatives from FireEye's Threat Intelligence DevOps team outlined their transition from a PostgreSQL-based system to a more robust architecture using JanusGraph with ScyllaDB as the storage engine, which enhanced their ability to process complex threat intelligence data at scale. The new system, built on an open-source graph database model, significantly improved query response times and allowed for more efficient data organization and threat analysis, enabling FireEye to handle massive volumes of data with nodes and edges representing various threat elements. This architecture, deployed in a secure AWS Virtual Private Cloud with components like NGINX web servers and Elasticsearch, provided high availability and disaster recovery capabilities, ultimately reducing operational costs by allowing the decommissioning of unnecessary nodes due to ScyllaDB's efficiency.