July 2019 Summaries

ScyllaDB University offers an engaging learning experience through its online courses, including the recently launched "Mutant Monitoring System" course, designed to teach administrators and developers how to effectively use ScyllaDB within their applications to manage large-scale data challenges. The course combines educational content with a narrative about saving Earth from a mutant invasion, covering key aspects such as setting up a ScyllaDB infrastructure, managing multi-data center replication, monitoring system performance, and implementing Java clients. Participants are encouraged to complete lessons, including hands-on labs and quizzes, to earn points and have a chance to win a ScyllaDB t-shirt. The course aims to equip learners with practical skills in data consistency, backup and restore operations, and advanced data analysis using tools like Apache Spark, fostering a comprehensive understanding of ScyllaDB's capabilities in large-scale data processing environments.

ScyllaDB Manager 1.4.2 is a maintenance release primarily focused on fixing bugs from the 1.4 release, enhancing the overall functionality of the ScyllaDB management system, which automates maintenance tasks on ScyllaDB clusters. Key improvements include resolving issues such as failing to access the store certificates directory when using Docker and providing a more informative error message when a connection to a ScyllaDB node fails. Additionally, it updates the Manager Go driver from version 1.0.1 to 1.2.0 and incorporates a token-aware policy for connecting to ScyllaDB cluster backends, aimed at improving user experience and system reliability for both ScyllaDB Enterprise and open-source users, with a limitation of five nodes for the latter.

OpenNMS Newts is a time-series database embedded within the OpenNMS platform, designed for network monitoring and built on ScyllaDB, which offers high-performance and scalable storage solutions. Initially relying on Cassandra, the Newts team discovered that ScyllaDB provided nearly double the throughput without significant changes, making it more efficient for their needs. Newts was developed to handle the exponential growth in datasets by optimizing for write performance, crucial for time-series metrics where data is often written more than read. Key features of Newts include its capacity for late aggregation, which reduces storage bottlenecks by aggregating data upon retrieval rather than during storage, and its ability to graph network data over time. Hosted on GitHub and licensed under AGPL v2.0, Newts is available for OpenNMS users aiming to scale their time-series data infrastructure beyond a single host, and it has been in use since 2015.

The upcoming ScyllaDB Open Source 3.1 release introduces support for local secondary indexes alongside the existing global indexing, providing users with more flexible query options tailored to their specific needs. While global indexes are beneficial for scaling across nodes with more complex writes, local indexes optimize queries that are partition-local, reducing latency by ensuring that the data retrieval occurs on the same node. The blog post discusses use cases for both indexing types, demonstrating that local indexes can significantly enhance performance for queries that are restricted to a single partition. The article also highlights the implementation details and performance implications, revealing that local indexes can outperform global ones in specific scenarios due to reduced inter-node communication. ScyllaDB's indexing strategy leverages materialized views, allowing for further optimization and flexibility in query execution, with future enhancements anticipated to enable more sophisticated indexing options.

ScyllaDB Enterprise has introduced encryption-at-rest in its 2019.1.1 release to enhance data security, especially in multi-tenant and hybrid deployments. The platform emphasizes a holistic security approach, which includes staying updated with the latest software versions, enabling authentication internally, and implementing Role-Based Access Control (RBAC) to enforce the principle of least privilege. ScyllaDB supports transport layer encryption (TLS) to protect data both from client applications and between nodes within the cluster. To minimize network exposure, the database should be kept off the public internet and only trusted clients should have access to specified network ports. Additionally, the auditing system in ScyllaDB Enterprise allows for the tracking and reporting of system activities, which is crucial for regulatory compliance and security oversight. The article highlights that while there is no single solution for data security, ScyllaDB Cloud can automatically cover most of these security measures for users.

SkyElectric, a company addressing unstable power grids in developing countries like Pakistan, India, Bangladesh, and Nigeria, utilizes ScyllaDB's NoSQL database to enhance its smart energy platform's performance and scalability. Initially, SkyElectric faced challenges with MySQL as their data needs grew rapidly, leading them to explore Apache Cassandra and Riak TS before settling on ScyllaDB due to its operational ease, speed, and scalability without the need for extensive tuning. Their transition to ScyllaDB resulted in significant performance improvements, with query times reduced from minutes to milliseconds, and allowed the team to focus on developing additional features such as a search engine and a states engine. The company, which engineers all its products in-house and provides continuous monitoring for its customers, plans international expansion and attributes much of its operational peace of mind to the reliability and support offered by ScyllaDB, thereby ensuring 24/7 sustainable and affordable energy availability.

ScyllaDB Enterprise has introduced encryption at rest in its 2019.1.1 release to enhance data security for persisted data on disk, including SSTables and commit logs. While ScyllaDB already supports data in transit encryption, the new feature allows for per-table and per-node data at rest encryption using any symmetric key algorithm supported by OpenSSL, providing users with more control over encryption keys as opposed to relying solely on cloud vendor solutions. The system uses local file key storage by default, with future support planned for replicated and KMIP key storage. This approach enables users to encrypt different tables with distinct keys, manage multi-tenant data access, and maintain data security even when data is moved across different volumes. Transparent data encryption ensures minimal CPU overhead and no additional disk footprint, though it requires careful management of encryption keys and file permissions to prevent data loss and ensure data safety against various types of attacks.

ScyllaDB Enterprise 2019.1.1 is a minor yet significant update to the ScyllaDB Enterprise 2019.1 branch, aimed at enhancing security and performance for its users. This release introduces encryption at rest, allowing on-disk data such as SSTables, commit logs, and batch logs to be encrypted, though centralized key management is not yet available. Additionally, ScyllaDB Enterprise is now accessible as a Docker instance and as an EC2 AMI, expanding its deployment options. A new BYPASS CACHE clause has been added for SELECT statements, optimizing performance for range scans by bypassing the cache for data that is unlikely to be accessed frequently. The update also addresses several bug fixes, including issues with the "mc" SSTable file format, range scan anomalies, and the TimeWindowCompactionStrategy, aimed at minimizing potential performance hindrances.

DBA Appreciation Day, held annually on July 5th, honors the crucial yet often overlooked role of database administrators, who manage complex database systems and ensure their continuous operation amidst evolving industry demands. Federico Campoli, the founder of DBA Appreciation Day, reflects on the historical perception of DBAs as magicians of technology and highlights the shift in database management towards abstraction, which has wrongly diminished the perceived need for DBAs. Despite this, Campoli believes that the role is more critical than ever, especially as databases become more sophisticated and cloud-based, and envisions it evolving into a data reliability engineer role. The inspiration for the event stemmed from personal experiences of unrecognized hard work, and it aims to balance organizational trust and recognition of DBAs' contributions. The future hope is for the event to gain wider recognition and appreciation for DBAs globally, supported by organizations like ScyllaDB, which emphasize the importance of making database management more manageable for these professionals.

ScyllaDB Open Source 3.0.8, released by the ScyllaDB team, is a bugfix update for the ScyllaDB 3.0 stable branch, maintaining backward compatibility and supporting rolling upgrades. This release addresses several issues, including an infinite loop in NodeJS driver paging indexes, clustering key restriction errors in secondary indexes, and complications with materialized view updates when a node fails. It also resolves errors such as partition key index failures and incorrect handling of empty key maps in mc format SSTable compaction, along with preventing core dumps from attempts to stop a disabled view builder.