August 2017 Summaries

The 2017 Q3 Apache Cassandra User Survey conducted by ScyllaDB gathered insights from 70 expert users managing over 200 clusters and 600TB of data to understand how Apache Cassandra is utilized in production. Key findings reveal that performance, tuning, and administration are significant challenges, making up 70% of issues faced by users. Despite these challenges, more than 60% of users prefer Amazon Web Services for infrastructure, while Open Source deployments surpass DataStax in popularity. DataStax OpsCenter emerges as the leading monitoring tool, though no clear leader dominates the community. The survey highlights that more than half the deployments use Cassandra Version 2.x, with Linux packages and Docker being the favored deployment methods. Java is the predominant programming language for application development, and SSD is the preferred storage type. The survey aims to offer the Cassandra community a benchmark for making informed database decisions, with nearly half of respondents utilizing multi-tenant deployments and a significant number managing operations across multiple data centers.

Phillip Tribble's blog post explores the synergy between ScyllaDB, a high-performance NoSQL database, and AWS EC2 Spot Instances managed by SpotInst's Elastigroup, highlighting both performance and cost benefits. ScyllaDB, known for its compatibility with Apache Cassandra, leverages C++ and the Seastar framework to outperform Cassandra significantly in throughput and latency. The article discusses the challenges of maintaining stateful services with spot instances due to their potential for abrupt termination, which is mitigated by Elastigroup's predictive algorithms and stateful features that ensure service continuity by preserving IP addresses, data volumes, and configurations. The post also details the configuration of ScyllaDB clusters across multiple data centers and racks to enhance data availability and reliability, using SpotInst's features to seamlessly handle spot market fluctuations. The blog emphasizes that this approach offers a robust combination of performance optimization and cost reduction, inviting readers to explore a free trial of the SpotInst platform.

At the NoSQL & NewSQL Database Meetup held at the AWS Loft in NYC, mParticle shared their experience transitioning from Apache Cassandra to ScyllaDB to handle their vast data processing needs, which include managing 50 billion monthly messages and a data volume of 150TB added to Amazon S3. Initially using Apache Cassandra for its high read and write throughput and scalability, mParticle encountered challenges such as costly commercial support, performance issues, and the need for extensive human resources for maintenance. ScyllaDB's compatibility with Apache Cassandra allowed mParticle to switch without any code changes, providing them with a more efficient solution due to its lower latency, better self-tuning capabilities, and responsive support team, which was praised for their expertise and dedication. The transition was seamless, with no service disruptions, and resulted in improved performance with ScyllaDB's enhanced throughput and lower compaction backlog.

Phillip Tribble's blog post features an interview with Glauber Costa, a Principal Architect at ScyllaDB, who previews his talk at the ScyllaDB Summit 2017. Costa discusses the advancements made in ScyllaDB, particularly in achieving consistent high percentile latencies, an area where the database has built a strong reputation. He highlights the implementation of "Workload Conditioning" algorithms designed to manage memory buffer challenges and ensure predictable latencies, even when faced with high loads. Costa emphasizes the importance of these improvements for both start-ups and enterprise users, noting that ScyllaDB 2.0 incorporates a year's worth of enhancements. The event, scheduled for October 24-25 in San Francisco, promises a variety of technical talks, a roadmap presentation, and a hands-on workshop, providing a comprehensive overview of ScyllaDB's capabilities and future direction.

Gocqlx is an extension to the Go ScyllaDB/Apache Cassandra driver Gocql, designed to enhance developer productivity without compromising query performance. Inspired by Sqlx, Gocqlx addresses several issues present in Gocql, such as the lack of named query parameters and the complexity of scanning rows into structs. It offers builders for creating queries, support for named parameters, and simplified binding of query parameters from struct fields or maps. Additionally, Gocqlx provides convenience functions for data retrieval, significantly streamlining operations like loading rows into memory. It is optimized for speed, utilizing the Reflectx package for cached reflections, and demonstrates superior performance in benchmarks compared to raw Gocql by reusing memory for binding values. Gocqlx's design allows it to coexist with Gocql, leveraging its strengths while offering a more flexible and maintainable approach to database interactions, all under the Apache License 2.0.

Raphael S. Carvalho, a computer programmer at ScyllaDB, is passionate about open-source software and kernel programming, contributing significantly to projects like Syslinux and MultiFS. At ScyllaDB, he has focused on SSTable compaction handling and developed the Time Window Compaction Strategy, which is considered superior to the DateTieredCompactionStrategy. Carvalho enjoys enhancing products with his programming skills and shares insights into his work and preferences in a recent interview on Coderintros.com, where he discusses his favorite programming language, his most impactful project, and his general views on coding.

In 2015, Intel and Micron introduced 3D XPoint, a non-volatile memory storage device that promises to be 1000 times faster than NAND, using electrical resistance and offering better endurance with a wear leveling of 30 full drive writes per day for five years. Intel's Optane SSD DC P4800X, based on this technology, is designed for PCI Express or NVMe slots in servers and boasts low latency and high throughput, making it suitable as both a memory cache and persistent storage. Performance tests show that Optane drives excel in low queue depth scenarios, significantly enhancing CPU utilization and offering consistent ultra-fast service, which can reduce total cost of ownership and expand capabilities for data centers. The technology supports near real-time data access with sub-millisecond read and write latencies, addressing challenges like cold-cache warming and data resiliency.

Phillip Tribble's blog post explores the integration of ScyllaDB and Elasticsearch to perform data mining on Twitter data, using Docker containers to set up a scalable and efficient system. By leveraging Twitter API credentials, the setup involves feeding Twitter data into ScyllaDB, with the possibility of exporting it to Elasticsearch for advanced analytics and visualization using Kibana. The demonstration highlights the architecture consisting of multiple nodes for ScyllaDB and Elasticsearch, and provides guidance on using Docker to manage the process. With step-by-step instructions, the tutorial shows how to query and visualize data, underscoring the combined power of ScyllaDB and Elasticsearch in handling data and analytics needs, while emphasizing the simplicity of setting up this system for both testing and production environments.

ScyllaDB employs chunk-based compression for SSTables, with a default chunk size of 4kB, which can impact performance based on the workload type. Larger chunk sizes are beneficial for write-heavy systems as they reduce disk bandwidth usage, while smaller chunks are advantageous for read-heavy systems, particularly those with single-partition reads, as they minimize unnecessary disk I/O and improve latency. The trade-off lies in the balance between read and write demands; adjusting chunk size can either optimize disk usage or increase overhead depending on whether reads or writes dominate. The chunk size can be altered using the ScyllaDB command line, and while smaller chunks increase memory usage due to more metadata, they can lead to significant disk bandwidth savings in read-mostly workloads. Additionally, larger chunks can offer better compression ratios, but users with very low chunk size requirements might consider disabling compression. Overall, selecting the appropriate chunk size requires understanding the specifics of the workload to optimize performance effectively.