March 2013 Summaries

Secondary indexes in Cassandra, introduced in version 0.7, allow data access using attributes other than the row key. They use an auxiliary column family to model an inverted index of values from a primary column family. The SecondaryIndex interface is an extension point for alternative implementations. However, secondary indexes add complexity as they need to be kept in sync with primary data. In Cassandra 1.2, the read-before-write requirement was removed by writing new index entries at the same time as updating primary data and deleting old entries lazily at query time. This led to performance improvements. To ensure consistency between primary data and secondary indexes, a RowMutation is received, and if any columns being mutated are configured with secondary indexes, additional work is required. The solution involves pushing updates to secondary indexes down the stack and implementing read-repair for indexes. This resulted in an ~11% improvement in write throughput.

Setting up a new DataStax Enterprise cluster on Amazon's cloud can be done in about ten minutes using DataStax OpsCenter. First, install DataStax OpsCenter Enterprise on an existing EC2 machine and ensure its interface parameter is configured properly in the config file. Next, use OpsCenter's Create Cluster option to create a new database cluster with desired specifications such as number of nodes and software components like Cassandra, Hadoop, and Solr. Finally, click the Build Cluster button, which will automatically set up the cluster. The actual time for this process may vary depending on factors such as instance spin-up time and download speed.

DataStax Enterprise (DSE) 3.0 and above supports data auditing on a database cluster, allowing administrators to understand user activity such as who looked at what/when and who changed what/when. Auditing is implemented via the log4J mechanism, providing flexibility in controlling what is audited, where the data is written, and how it is presented. It is disabled by default and can be enabled by editing the log4j-server.properties file. The granularity of auditing can be controlled using the ActiveCategories setting. Audit logs can be written to a table/column family with DSE's built-in encryption for sensitive data. Hadoop and Solr user activities can also be audited in DataStax Enterprise.

DataStax Enterprise 3.0 introduces transparent encryption (TDE) as a new security feature. TDE encrypts data at rest, protecting sensitive information such as HR records and credit card details. It is applied at the table or column family level during creation time using various encryption algorithms like AES 128. Once specified, no further action is needed for developers or administrators. For existing tables, an ALTER command is used followed by a nodetool scrub or flush command to update SStables with encryption. TDE can also be applied to Hadoop data in DataStax Enterprise but not to Solr indexes. Limitations include the need for a subscription for production deployments and no support for encrypting all Solr data.