February 2014 Summaries

The upcoming release of Cassandra 2.0.6 introduces new features to CQL including batch conditional updates and static columns. Batch conditional updates allow grouping multiple conditional updates in a single batch, provided they apply to the same partition. Static columns are special columns shared by all rows of the same partition, offering possibilities for maintaining invariants and optimizations. Additionally, Cassandra 2.0.6 introduces a new syntax for more general "slicing" over multiple clustering columns within a single partition.

Repairs are crucial for maintaining data consistency in a Cassandra cluster, especially when frequently deleting data. The nodetool repair command initiates the repair process on a specific node by computing a Merkle tree for each range of data on that node. Incremental repairs have been introduced in Cassandra 2.1 to persist already repaired data and only calculate merkle trees for sstables that haven't previously undergone repairs, making the repair process more efficient as datasets grow. The incremental repair process involves sending a prepare message, building merkle trees from unrepaired sstables, comparing trees, issuing streaming requests, and finally an anticompaction command to segregate repaired and unrepaired ranges into separate sstables. Full repairs remain the default, but incremental repairs can be opted into via the -inc option to nodetool repair.

DataStax Enterprise (DSE) 4.0 introduces an in-memory option for Cassandra, offering benefits such as multi-data center/cloud support and flexible data model while maintaining no single point of failure. This new feature allows users to assign data based on performance needs, including spinning disk for lesser workloads, SSDs for hotter data, and in-memory for the fastest response times within a single database cluster. DSE is one of the few NoSQL databases that can handle both big data scale and in-memory use cases, making it suitable for applications like eBay with massive big data workloads alongside low latency requirements. In-memory computing has become popular among IT staffs aiming to create fast online apps, but traditional relational database management systems (RDBMS) still suffer from rigid data models, multiple points of failure, and inability to truly scale out linearly. Cassandra with in-memory computing addresses these issues, offering continuous availability, linear scale performance, support for modern data types, and multi-data center/cloud availability that legacy RDBMS technology cannot match.

Cassandra 2.1 introduces User Defined Types (UDTs), allowing for more flexibility in data storage. UDTs can be nested arbitrarily, but currently cannot update only parts of a UDT value and act like blobs. Secondary indexes on collection columns are now allowed, with the option to index map keys or values separately. However, simultaneous indexing of both keys and values is not yet possible.

The text discusses three broad categories of storage in AWS: instance store (ephemeral), EBS (Elastic Block Storage), and S3. EBS is mountable storage that can be mounted as a device to an EC2 instance, while S3 is a web-based storage service replicated across regions. Both are persistent and exist independent of an EC2 instance. EBS is further refined into two categories: Standard and Provisioned IOPS (PIOPS). PIOPS is tunable and predictable, with the desired number of IOPS per volume guaranteed. Instance store or ephemeral storage is physically attached to an EC2 instance but evaporates when the instance is shut down. AWS has started offering beefier machines with "direct-attached" storage that is not a shared resource like instance store and can be faster and more reliable.

Benchmarking Cassandra against other systems can lead to valuable insights, but many results are less useful due to preventable errors. Some common mistakes include using VMs with noisy neighbors, shared storage that becomes a bottleneck, and inadequate low-level operations like random reads. Additionally, benchmarking with small datasets or failing to reset the cluster between runs can lead to misleading results. To ensure accurate benchmarks, it's crucial to use established load generators, configure disks properly, allow JVM warmup, and follow best practices for performance tuning.