July 2017 Summaries

The blog post critically examines the use of external caches in front of databases, highlighting both their potential benefits and drawbacks. It argues that while caching may initially appear to be a straightforward solution for improving database performance without extensive reconfiguration, it introduces several issues such as increased latency, additional costs, decreased availability, and added complexity in application management. The post notes that external caches can disrupt the internal caching mechanisms of modern databases, leading to inefficiencies and security vulnerabilities. Through examples like ScyllaDB, which effectively utilizes internal caching and minimizes reliance on external caches, the article suggests that leveraging built-in database caching capabilities can often be more efficient. It presents the perspective that many challenges associated with external caches can be mitigated by optimizing internal database caching, thus questioning the necessity of external caching solutions in most scenarios.

ScyllaDB 1.7.4 is a bugfix release for the ScyllaDB 1.7 stable branch, maintaining backward compatibility and supporting rolling upgrades, with particular attention needed for changed metric names when upgrading from version 1.6.x. The release addresses a specific bug introduced in version 1.7.3, where large cluster setups could enter an infinite loop. Users are encouraged to review the ScyllaDB 1.7 release notes for a smooth transition and to report any problems encountered during the upgrade process.

ScyllaDB has introduced Materialized Views as an experimental feature in its upcoming 2.0 release, aiming to enhance data organization by automating the management of different table views. This feature, compatible with Apache Cassandra's implementation, allows data to be indexed using a global index, thus improving scalability compared to local indexing methods like Secondary Index and SASI. Materialized Views enable applications to automatically maintain multiple views of the same data, simplifying the process and reducing the need for complex application logic to ensure data consistency across views. Although this feature is experimental and has limitations, such as incomplete data migration and issues with concurrent updates, it promises improved data consistency and development speed. Users are encouraged to test Materialized Views in non-production environments, as future releases will address current constraints.

ScyllaDB 1.6.6 is a bugfix release for the 1.6 stable branch, addressing several issues such as a minor memory leak caused by using User Defined Types (UDT) and unexpected exits following alterations to table replication factors. This release remains backward compatible and supports rolling upgrades, though users are encouraged to update to the more recent stable release, ScyllaDB 1.7.2. Notably, the update process from version 1.5.x to 1.6.6 requires attention to changes in metric names. The release also highlights performance impacts related to nodetool operations, like major compaction and cleanup, which could affect disk space and client request performance. Users are advised to report any issues encountered during the upgrade process.

ScyllaDB has released version 1.7.2, a bugfix update for the ScyllaDB 1.7 stable branch, ensuring backward compatibility and support for rolling upgrades. Users upgrading from version 1.6.x are advised to read the release notes due to changes in some metric names. This release addresses several issues, including the improper closure of Thrift connections, failures in the posix_net_conf.sh script during bonding interface configuration, and performance improvements in repair operations by iterating over subranges. Additionally, it fixes memory leaks linked to User Defined Types (UDTs), errors when dropping keyspaces with nested UDTs, crashes from truncating tables consecutively, blocking of the RPC server by large messages, and the potential for accessing freed memory leading to incorrect results. Other fixes involve resolving segfaults during nodetool cleanup, exceptions thrown by sstabledump when dealing with UDTs, and errors reported by node_exporter when monitoring disk usage metrics. Users are encouraged to report any further issues encountered.

Christopher Quinones, the project lead for Compose for JanusGraph, discusses the development and advantages of JanusGraph in an interview, highlighting its capabilities as a scalable graph database optimized for complex data interactions. JanusGraph, which is Apache TinkerPop-enabled, is particularly suited for applications like fraud detection and social media analysis, due to its ability to efficiently manage highly interconnected data. The interview touches on the decision to use ScyllaDB as a backend replacement for Cassandra, noting ScyllaDB's advantages like user space networking and a share-nothing thread model. Quinones shares his journey from a software developer focused on automating deployments to becoming proficient with graph databases, emphasizing the importance of understanding data modeling as nodes and relationships. He also refers to various resources and communities, such as Apache TinkerPop and JanusGraph Community, as valuable tools for learning and experimenting with graph data models.

In a survey conducted at the 2017 AWS Summit in San Francisco, researchers gathered insights from 117 developers about their adoption of NoSQL databases on Amazon Web Services (AWS). The survey revealed that over 90% of respondents use MySQL in production, with MongoDB, Postgres, Redis, and Apache Cassandra also being popular choices. Docker is utilized by more than 50% of participants in production, and 63% use various container solutions, highlighting the growing trend of containerization in database management. The primary challenges faced by developers include latency, management, and scalability, often due to the resource-intensive nature of Java-based NoSQL technologies. The survey also identified activity tracking, heavy read/write operations, and time series as the leading use cases for NoSQL databases, emphasizing the need for real-time data processing across multiple locations. The findings serve as a benchmark for understanding the criteria and results of NoSQL adoption among AWS users in the tech-centric San Francisco Bay Area.

ScyllaDB has made significant improvements in handling CPU-bound workloads by implementing function call batching inspired by the Staged Event-Driven Architecture (SEDA), which has resulted in a notable increase in throughput and efficiency. The process involved diagnosing performance bottlenecks using tools like Flame Graphs and Performance Monitoring Units (PMU) to analyze front-end latency issues, particularly instruction cache misses. By introducing execution stages at key points in the request processing pipeline, ScyllaDB was able to enhance instruction cache locality, thereby increasing instructions per cycle and reducing cache miss rates. This approach has shifted the performance bottleneck from the CPU front-end to other parts of the microarchitecture, without significantly affecting latency. While the current improvements have been substantial, further fine-tuning could yield additional performance gains, but such optimizations must be handled carefully to avoid increasing latency.