December 2018 Summaries

At the ScyllaDB Summit 2018, CEO and Co-founder Dor Laor delivered a keynote highlighting the evolution and future of ScyllaDB, a NoSQL database platform. Laor discussed the company's growth since its inception in 2015, including the release of ScyllaDB 1.0 and subsequent updates that have allowed it to achieve feature parity with Cassandra. Notable advancements include the introduction of ScyllaDB 3.0, which offers advanced features like SSTable format 3.0 and Materialized Views, as well as the launch of ScyllaDB Cloud on AWS. Laor emphasized ScyllaDB's competitive edge over other NoSQL databases, such as Amazon DynamoDB, citing lower costs and improved performance in specific scenarios. Additionally, the summit introduced the innovative per-user SLA feature, enabling efficient resource management for mixed workloads. Looking ahead, Laor outlined plans to enhance ScyllaDB's offerings further, focusing on lightweight transactions, improved performance, and ease of management, while expressing optimism for continued breakthroughs in the Big Data landscape.

The integration of ScyllaDB and Confluent, leveraging technologies like MQTT and Apache Kafka, offers a robust architecture for managing Internet of Things (IoT) deployments by addressing challenges related to data flow and scalability. This architecture uses MQTT for lightweight M2M communication and Kafka for high-velocity data ingestion, allowing for real-time data processing and storage in ScyllaDB, a scalable NoSQL database. The system is particularly suited for IoT applications that rely on streaming data, enabling real-time and batch analytics through ScyllaDB. The integration demonstrates a practical use case with sensors emitting data to Kafka, which is then processed and stored in ScyllaDB, showcasing the synergy between these technologies for building scalable, efficient IoT solutions. This approach is already employed by companies like General Electric and Nauto, providing cost-effective and reliable solutions for diverse applications such as smart grids, remote patient monitoring, and commercial fleets.

The blog post discusses the evolving landscape of software licensing in the open-source community, highlighting a trend toward more restrictive licenses as a response to pressure from major cloud providers like AWS. It mentions several companies, including Confluent and MongoDB, which have adopted more protective licensing models, such as the Confluent Community License and the Server Side Public License (SSPL), to safeguard their intellectual property from being exploited by cloud services. This shift is seen as a departure from the collaborative spirit traditionally associated with open-source projects, potentially creating silos and limiting contributions from smaller as-a-service vendors. The author argues that while these changes may be a logical response to the dominance of large cloud providers, they ultimately represent a step backward for the industry. The post suggests that a more collaborative approach between open-source vendors and infrastructure-as-a-service providers, focusing on shared benefits and contributions, would be more beneficial, and encourages end users to support vendors who align with long-term ecosystem values.

The blog post provides a comparative analysis between ScyllaDB and Amazon DynamoDB, focusing on performance, cost, and flexibility. It frames the comparison as a David versus Goliath scenario, given Amazon's dominance in cloud infrastructure against the younger ScyllaDB. The analysis highlights that ScyllaDB offers significantly lower latencies and costs compared to DynamoDB, particularly excelling in scenarios involving high-throughput operations and non-uniform data distributions. The tests, conducted using the Yahoo! Cloud Serving Benchmark (YCSB), reveal that ScyllaDB handles workloads more efficiently with lower resource utilization and without throttling issues evident in DynamoDB. Additionally, ScyllaDB's open-source nature and absence of cloud vendor lock-in provide users with greater flexibility and cost-effectiveness, especially when considering total cost of ownership and the ability to run across multiple cloud platforms or on-premises. The blog post concludes by advocating for ScyllaDB's superior performance and cost advantages, suggesting that despite its smaller market presence, it offers compelling benefits over DynamoDB.

ScyllaDB Manager 1.3 has been released for ScyllaDB Enterprise customers, featuring a new Health Check function designed to address potential issues undetectable by the existing nodetool status or ScyllaDB Monitoring Stack dashboards. This function sends a CQL OPTION command to each node, assessing response times to determine node status, with a response under 250ms indicating an 'up' status. The results are accessible via the "sctool status" command, and a new alert in Prometheus Alert Manager notifies users when a node is considered 'down'. The release also integrates with the updated ScyllaDB Monitoring Stack 2.0, which includes health check reports on the Manager dashboard, enhancing centralized cluster administration and task automation capabilities. The upgrade guide emphasizes redefining scheduled repairs, and ScyllaDB Manager is available for download for a 30-day trial, promising future enhancements like rolling upgrades and recurrent backups.

In a talk at the ScyllaDB Summit 2018, Nadav Har'El discussed strategies for managing data ingestion in ScyllaDB, focusing on maintaining optimal write speeds without overwhelming the database. ScyllaDB uses flow control mechanisms to balance the rate of data writes, preventing server overload by using both throttling and admission control. The talk detailed how the database manages different workload types—batch and interactive workloads—by either delaying client responses or failing new requests when necessary. For tables with materialized views, ScyllaDB 3.0 introduces advanced flow control that includes a linear control function to adjust client delays, ensuring that the system's memory and processing resources are not exceeded. These mechanisms allow ScyllaDB to operate efficiently without requiring manual configuration from users, supporting autonomous database management. The blog post also highlighted a simulation tool used to demonstrate these flow-control algorithms, emphasizing the goal of achieving worry-free ingestion by automatically adjusting to write loads.