October 2017 Summaries

At the ScyllaDB Summit, a focus on performance and scalability was highlighted through various talks and demos by experts from diverse fields. Joseph Fullop and Miguel Martinez Pedreira discussed high-powered computing and data migration to Apache Cassandra/ScyllaDB, while Shlomi Livne offered insights into optimizing query performance. Jesse Haber-Kucharsky demonstrated ScyllaDB on Kubernetes, and Arash Rezaei emphasized performance tuning with Samsung's NVMe SSDs. Presentations covered a range of topics, including streaming ETL on Apache Kafka by Hojjat Jafarpour, stateful streaming with Apache Spark by Burak Yavuz, and ScyllaDB’s use in Page Context Categorization by Rocket Fuel. Twitter's Boaz Avital shared experiences with managing large storage clusters, and Raphael Carvalho and Nadav Har’El discussed compaction strategies. Zen.ly's migration from Elasticsearch was detailed by Jean-Baptiste Dalido, while Brian Hawkins explored time-series databases. Avi Kivity and Alexander Sicular addressed tools for performance understanding and migration strategies, with Eyal Gutkind focusing on optimization and cost savings. Glauber Costa and Shlomi Livne discussed repair, backup, and user-defined types, concluding with Pekka Enberg's talk on scalable secondary indexes. The summit's presentations, including emerging features and user interviews, are available online for further exploration.

ScyllaDB Summit 2017's first day featured a range of technical presentations and networking opportunities, highlighted by the announcement of ScyllaDB 2.0, which aims to become the first autonomous NoSQL database with self-tuning capabilities. Keynotes included insights from ScyllaDB's leadership on the company's vision, product roadmap, and recent acquisition of Seastar.io for future database-as-a-service offerings. Presentations from customers, such as mParticle's transition from Apache Cassandra to ScyllaDB, showcased significant performance improvements, while ScyllaDB’s engineers introduced new features like Materialized Views and Heat-Weighted Load Balancing. The event also included Ignite Talks and breakout sessions on a variety of topics, including performance tuning and using ScyllaDB with JanusGraph and large-scale data platforms. The day concluded with live demonstrations and networking, with plans to release presentation materials and videos online.

ScyllaDB announced its acquisition of Seastar.io's database-as-a-service technology from Network Redux, aiming to enhance its enterprise offerings by providing cloud-hosted ScyllaDB solutions on platforms like AWS, GCE, and Azure. This acquisition addresses the demand for scalable, cost-effective, and easily managed cloud databases, leveraging Seastar.io's capabilities to automate and secure the deployment of Apache Cassandra-compatible databases. A notable feature introduced is the ScyllaDB Test Drive, which allows users to quickly deploy a cluster on AWS for testing purposes, with the integration expected to fully materialize by 2018, promising a self-service product addition to ScyllaDB's portfolio. The new capabilities will be showcased at the upcoming ScyllaDB Summit 2017.

At the ScyllaDB Summit 2017, Michal Matczuk, a software engineer at ScyllaDB and a Go programming enthusiast, will discuss GoCQLX, an open-source project designed to enhance Go applications by providing tools over the Go Cassandra driver, GoCQL. The library aims to reduce boilerplate code, minimize bugs, and streamline database interaction by simplifying query writing, parameter binding, and result scanning. Michal's talk will cater to Go programmers and application developers, presenting both introductory and advanced use cases of GoCQLX. The ScyllaDB Summit in San Francisco will feature various technical talks and workshops, including insights into the ScyllaDB roadmap and best practices for utilizing ScyllaDB clusters.

At the ScyllaDB Summit 2017, Frank Ober from Intel is set to discuss the advantages of Intel's Optane SSDs, particularly focusing on how NVMe has evolved with 3D XPoint media into Optane Technology, which offers significant improvements over standard SSDs. Optane is highlighted as a key differentiator due to its balance and high performance in random write workloads, largely thanks to its unique memory design that allows faster access and synchronous IO capabilities. Ober emphasizes that Optane can reduce the dependency on DRAM for databases, offering a new tier of memory that enhances performance and flexibility at a lower cost. The technology is particularly beneficial for innovators and early adopters, as it allows for better memory utilization and supports high VM density and database-driven decision-making. The talk is aimed at developers, systems engineers, and decision-makers interested in improving I/O performance with advanced SSDs, and more information can be found on Intel's website.

ScyllaDB offers robust fault tolerance features designed to maintain data consistency and system operation even when one or more nodes fail. Key strategies include setting appropriate replication factors and consistency levels, which dictate how data is replicated across nodes and the number of nodes required to acknowledge read or write operations. ScyllaDB's approach aligns with the CAP Theorem, balancing consistency, availability, and partition tolerance to optimize performance and latency. The recently updated fault tolerance documentation provides insights into these concepts and offers guidance relevant not only to ScyllaDB but also to other distributed databases like Apache Cassandra and Amazon DynamoDB. Users can explore this information in the new documentation to better understand the architecture and operations of ScyllaDB, with resources available for both administrators and developers.

ScyllaDB 2.0, a major release of the open-source NoSQL database compatible with Apache Cassandra, introduces several enhancements aimed at improving performance and usability. This version offers experimental support for Materialized Views, which automates the process of querying the same record by multiple keys, and introduces a row cache feature that significantly boosts performance for large partitions. Additionally, the release includes Heat Weighted Load Balancing to optimize node selection, native support for EC2 i3 instances, and experimental CPU isolation for enhanced latency management. While the update deprecates older Cassandra drivers in favor of those compatible with Cassandra 3.0, it maintains backward compatibility with all ScyllaDB 1.x versions. Users are encouraged to upgrade from ScyllaDB 1.7.4 or later, and to consult the impact and known issues sections before proceeding. Notable fixes and new metrics are also part of this release, alongside improvements for CPU-bound workloads and DNS resolution capabilities.

Jean-Baptiste Dalido, the Head of Infrastructure at Zenly, shared insights into the company's migration from Elasticsearch to ScyllaDB, highlighting the challenges and successes encountered in the process. With a background in managing heavy-load databases and a preference for distributed systems, Dalido discussed the reasons behind choosing ScyllaDB, such as its robust performance and suitability for production use, which enabled Zenly to handle over 600,000 requests per second with minimal nodes. He emphasized the differences in data model tuning between ScyllaDB and Apache Cassandra and noted that ScyllaDB has become the sole database at Zenly, aside from Redis, which is still used for caching. Dalido's upcoming talk at the ScyllaDB Summit 2017 aims to illustrate the practical benefits of ScyllaDB and engage in discussions about its potential for various use cases.

Chris Bannister, the maintainer of the GoCQL package, which is a Golang driver for Apache Cassandra and ScyllaDB, discusses his work and upcoming presentation at the ScyllaDB Summit 2017. Bannister explains GoCQL's capabilities, such as automatically discovering data centers and distributing queries efficiently, and highlights the advantages of using Go for concurrency and error handling. He plans to explain how to use GoCQL to execute queries and provide insights into the driver's internal operations during his talk. Bannister emphasizes the importance of reliable testing and the continuous integration system he has set up for GoCQL to ensure its robustness. While acknowledging that GoCQL does not support all features present in the Java driver, he notes that GoCQL is designed to be a low-level, yet functional driver for most situations, with external libraries available for additional features. Bannister also mentions ongoing improvements, such as working on the v5 binary protocol and refining retry policies, while expressing his reservation about integrating more complex functionalities directly into the driver.

Nadav Har'El, a software developer at ScyllaDB, discusses the importance of selecting the right compaction strategy to optimize workload performance on a ScyllaDB cluster in an interview ahead of the ScyllaDB Summit 2017. With a background in software development and open-source projects, Har'El highlights his early programming experiences and contributions to ScyllaDB and the Seastar programming framework. He plans to address the various compaction strategies used by ScyllaDB and Apache Cassandra at the summit, introducing a new strategy unique to ScyllaDB that aims to blend the best features of existing methods while avoiding their drawbacks. This new strategy is expected to cater to a wider range of workloads, providing automatic tuning to balance compaction speed and query performance, unlike the traditional user-controlled approach in Apache Cassandra. Har'El emphasizes the benefits of "workload conditioning" in ScyllaDB, which automatically adjusts the pace of compaction to maintain optimal request latency.

Server application developers often focus on network I/O, but database developers must also consider file I/O, particularly when choosing access methods on Linux servers. The text outlines four primary methods: traditional read/write, mmap, Direct I/O (DIO), and asynchronous direct I/O (AIO/DIO), each with distinct tradeoffs regarding cache control, copying, MMU activity, I/O scheduling, thread scheduling, I/O alignment, and application complexity. Traditional methods rely heavily on the kernel for caching and scheduling, while DIO and AIO/DIO offer more control to the application, although they increase complexity. ScyllaDB opts for AIO/DIO to maximize performance and control, utilizing the Seastar framework to manage the complexity and optimize I/O operations such as compaction and queries. This approach allows ScyllaDB to bypass the kernel's limitations, improve cache management, and align small reads for efficiency, showcasing the potential of directly driving NVMe drives in the future.

In an interview ahead of the ScyllaDB Summit 2017, Arash Rezaei, a Senior Performance Architect at Samsung Semiconductor, discusses the benefits of Samsung's Z-SSD drives for ScyllaDB users, emphasizing their potential to bridge the performance gap between in-memory and in-storage systems. He explains that his upcoming talk will focus on performance characterization and tuning of ScyllaDB on Samsung's NVMe SSDs, highlighting how Z-SSDs can enhance system performance while reducing costs. Rezaei notes that Z-SSD, a new NAND flash-based product, offers ultra-low latency and is suitable for high-capacity caching, NoSQL databases, and real-time analytics, making it a valuable asset for cloud computing infrastructures. The ScyllaDB Summit, taking place in San Francisco on October 24-25, promises to offer insights into these advancements alongside other technical talks and workshops.

Intel Optane drives are set to revolutionize data centers by providing lower latencies and higher throughput, benefiting ScyllaDB users by offering the speed of an in-memory database with data persistence. In an interview, ScyllaDB's Solutions Architect, Tomer Sandler, discussed his role, which involves testing and demonstrating use cases, conducting training, and writing technical content. Tomer, alongside Frank Ober from Intel, will present at ScyllaDB Summit 2017 about the performance enhancements of using Intel Optane drives with ScyllaDB, highlighting their potential to improve compaction performance and lower total cost of ownership compared to traditional SSDs. The discussion also addressed the market dynamics of new technologies and their adoption in enterprise settings. ScyllaDB Summit, held in San Francisco, offers additional technical talks and workshops to help attendees maximize their ScyllaDB clusters.

Phillip Tribble's interview with Amiram Shachar of Spotinst highlights how organizations can significantly reduce infrastructure costs by utilizing ScyllaDB on EC2 Spot Instances, enhanced by Spotinst's predictive technology. As the founder and CEO of Spotinst, Amiram shares his journey from leading datacenter architecture teams in the Israeli Defense Forces to starting Spotinst as part of an academic research initiative. His upcoming talk at the ScyllaDB Summit 2017 will focus on demonstrating how Spotinst's advanced algorithms and stateful application technology can mitigate the unpredictability of EC2 Spot instances, thereby allowing companies to run a broad range of applications with improved elasticity and performance while achieving up to 80% cost savings. The session is aimed at CTOs, VP R&Ds, DevOps, and IT Managers who are interested in infrastructure cost-saving strategies, and will include a live demo showcasing the integration of ScyllaDB with Spotinst. Attendees can also explore the wider ScyllaDB Summit agenda, which features technical talks, a roadmap presentation, and a hands-on workshop to maximize the use of ScyllaDB clusters.