June 2015 Summaries

Apache Cassandra introduces Materialized Views in version 3.0 as a feature that handles automated server-side denormalization, removing the need for client side handling of this process. This feature ensures eventual consistency between the base and view data, allowing for very fast lookups of data in each view using the normal Cassandra read path. Materialized views maintain a correspondence of one CQL row each in the base and the view, and can be used to track high scores for players of several games, among other applications. They are created by providing a simple select statement and the primary key to use for this view.

The latest maintenance releases of DataStax Community Edition, versions 2.1.7 and 2.0.16, are now available for download from Planet Cassandra. These updates do not have any version-specific upgrade notes but users should review the relevant NEWS entries if upgrading from an older release. The CHANGES.txt and NEWS.txt files provide detailed information on the changes made in each version.

The text discusses how to improve the performance of Python applications working with large datasets, which often become CPU bound due to serialization and deserialization processes. It suggests using the multiprocessing package from the Python standard library to distribute work among multiple processes, allowing applications to utilize multiple CPUs. The author provides a detailed example demonstrating how to use multiprocessing with the DataStax Python Driver to achieve higher throughput. They also highlight some trade-offs and considerations when using this pattern, such as overhead costs and latency sensitivity.

The commit log implementation of Apache Cassandra has been significantly improved in version 2.1, with multithreaded and memory-mapped writing reducing log overheads and improving throughput. In version 2.2, compression was added to the commit log, further enhancing performance by reducing disk transfer and space requirements at the expense of additional CPU processing. The segment reuse feature has been removed in this version to reduce page cache pressure and improve real-world log performance. Compression is configured using the commitlog_compression parameter in cassandra.yaml, with LZ4Compressor recommended for its reasonable compression rates and high throughput. In testing, compression resulted in a 6-12% improvement in write performance and smoother operation under heavy writes.

The text discusses a planned release strategy for Apache Cassandra, an open-source distributed database management system. It mentions that the upcoming releases will be split into two parts - version 2.2 to be released in July and version 3.0 in September. The new versions will include features such as Windows support, commit log compression, JSON support, role-based authorization, bootstrap-aware leveled compaction, user-defined functions, major storage engine rewrite, and materialized views. The reason for this split is to prevent blocking of features that are already complete while waiting for the new storage engine (8099). Releasing them now as 2.2 reduces risk for users and allows independent stabilization. After 3.0, a monthly release cycle will be implemented with even releases including both bug fixes and new features, and odd releases being bug-fix only. This approach is referred to as "tick-tock" releases, inspired by Intel's policy of changing process and architecture independently. The primary goal of this strategy is to improve release quality. The current major "dot zero" releases require five or six months to make them stable enough for production due to complex interactions between new features. Tick-tock releases will reduce the number of features in each version, improving the ability to quickly track down any regressions and preventing situations like the one with 8099 delaying everything else. The text also mentions efforts being made to make trunk "always releasable" by extending continuous integration server capabilities and investing in more test infrastructure and procedures. Backwards compatibility policy will be extended to cover all 3.x releases, allowing seamless upgrades from one version to another. Under normal conditions, no extra upgrade requirements or removal of deprecated features will occur until version 4.0. The text concludes by stating that after the release of 2.2 and 3.0, versions 2.0 and 2.1 will reach end-of-life as planned. The new strategy aims to deliver production-level stability while maintaining a continuous flow of feature updates and bug fixes.

Cassandra 2.2 introduces an extension to CQL that makes working with JSON documents easier. The SELECT and INSERT statements now include a JSON-focused variant, and two new native functions have been added for conversion between JSON and other formats. This feature ensures users continue to work with data in a type-safe, schema-enforced way. Cassandra types that have a sensible native JSON equivalent are accepted, while those without clear equivalents require string representations matching the normal CQL literal format. Lists, sets, tuples, and user-defined types can be represented by JSON maps or lists. The SELECT statement has also been extended to support retrieval of rows in a JSON-encoded map format. Additionally, new functions toJson() and fromJson() have been introduced for single column operations. Overall, these improvements make it easier to work with JSON documents while maintaining the benefits of schema enforcement.

Apache Cassandra 2.2 introduces an exciting feature for users of LeveledCompactionStrategy (LCS). When bootstrapping a new node, LCS requires re-leveling the data which can take days to complete due to many pending compactions. However, CASSANDRA-7460 addresses this issue by re-using the level of each source SSTable streamed to the new node. This means that instead of days of compacting after adding a new node, there will be zero compactions. The process involves regular cluster writes during bootstrap ending up in level 0 upon flush and passing the level as part of streaming is only used for bootstrapping or node replacement scenarios.

The Java driver team has released version 2.2.0-rc1, which brings parity with Cassandra 2.2. This release introduces four new types and support for date-only and time-only values. Additionally, user-defined functions are now exposed as metadata. Unset values do not overwrite previously inserted ones, and warnings from the server are now sent back with the response instead of just being logged server-side. The driver is available from Maven and the downloads server, but this release candidate should not be used in production until the final release.

This blog post delves into deeper detail on the inner workings of Apache Spark and how to shape your application to take advantage of interactions between Spark and Apache Cassandra. It covers key components of Spark, such as its four processes, executor JVMs, heap memory allocation, and RDDs. The post also discusses troubleshooting connections between the driver and executors, minimizing shuffles, caching RDDs, leveraging Cassandra's advantages within Spark, and using metrics to monitor throughput to and from Cassandra.

The CQL WHERE clause differs from SQL due to the distributed nature of Cassandra data and its aim to prevent inefficient queries. In Cassandra, two types of columns have a special role: partition key columns and clustering columns. Together, they define your row primary key. Partition key columns are used to spread data evenly around the cluster, while clustering key columns cluster the data of a partition for efficient retrieval. The WHERE clause supports different sets of restrictions depending on the type of query (SELECT, UPDATE, or DELETE) and the column being restricted. Cassandra requires that all partition key columns be restricted in SELECT queries unless secondary indexes are used. Clustering columns support various operators in single-column and multi-column restrictions. Secondary index queries allow for additional filtering on non-indexed columns using =, >, >=, <=, <, CONTAINS, or CONTAINS KEY restrictions. UPDATE and DELETE statements require all primary key columns to be restricted, with only specific operators allowed.