July 2014 Summaries

The cassandra-stress tool has been extended in Cassandra 2.1 to support stress testing arbitrary CQL tables and queries on that table. This makes testing data models simpler and helps users understand the tradeoffs they are making in their data model and settings. The new cassandra-stress now supports a YAML based profile that is used to define your specific schema with any potential compaction strategy, cache settings, and types you wish. The tool covers a lot of use cases but there are some things it can't do, which will be addressed in future releases.

Cassandra 2.1 introduces incremental repairs, which makes repair operations more efficient by not re-repairing already repaired data. Anticompaction is a key concept that enables incremental repairs. It involves splitting an SSTable into two - one containing repaired data and another with unrepaired data. This separation allows for adjustments to compaction strategies, such as size tiered compaction and leveled compaction. The process of migrating to using incremental repairs requires careful handling of SSTables and their levels during the transition period.

In Apache Cassandra, when using "nodetool repair", it is crucial to think in terms of token ranges rather than individual tokens. The default options for "nodetool repair" initiate a repair for every token range owned by the node. However, using the "-pr" option allows each node to pick a subset of its token range for repair, ensuring that every token range is only repaired once across all nodes in the cluster. This becomes especially important when dealing with multiple data centers and large numbers of nodes. Additionally, Cassandra 2.1 introduced incremental repair, which further optimizes the repair process by only performing synchronization on previously unrepaired data.

The text discusses the enhancements in Apache Cassandra version 2.1, particularly focusing on read and write performance improvements. It mentions a 75% increase in read performance compared to CQL 2.0 and a 160% increase over Thrift. Write performance also saw a significant boost of 95% over CQL 2.0 and 150% over Thrift. However, the write performance is inconsistent due to commitlog disk being the bottleneck on a 32-core VM. The team plans to prioritize commitlog compression and support for multiple commitlog volumes to address this issue.

The text discusses the importance of moving data structures off the Java heap to native memory due to increasing dataset sizes and limitations on JVM heap size. In Cassandra 2.1, improvements are made to reduce the footprint of memtables with a new configuration option called `memtable_allocation_type`. Two options are provided: `heap_buffers` (default) and `offheap_objects`, which is expected to become the default in version 3.0. Laboratory tests show that enabling `offheap_objects` results in approximately 5% faster writes due to less frequent flushing, leading to larger SSTables and reduced compaction needs. Reads remain unchanged but may vary under different workloads. Future enhancements are planned to better test the performance of memtables with more demanding scenarios.

DataStax Enterprise (DSE) 4.5 introduces integration with external Hadoop vendors such as Cloudera and Hortonworks, enhancing its analytical capabilities. This feature, known as Bring Your Own Hadoop (BYOH), allows users to install certain Hadoop components from a preferred vendor on a DSE cluster to run analytic jobs directly on Apache Cassandra™ data. The external Hadoop integration is aimed at customers looking for an integrated analytics solution and can be used with the new Apache Spark™/Shark option available as part of DSE 4.5. BYOH nodes are not part of DSE's analytic datacenter, and best practice recommends deploying them in a virtual data center to avoid interference between Cassandra (OLTP) resources and BYOH analytic workloads.