May 2014 Summaries

Masterless databases like Cassandra are designed to handle any failure without interruption, including node failures, garbage collection pauses, or slow disks. These failures are indistinguishable from network partitions in the system. This fault-tolerant design also helps lower latency as there is no artificial bottleneck that causes spikes during hiccups. Cassandra's faster overall performance and fault-tolerant design allow it to achieve low latency, such as serving 99th percentile of reads under 10 ms in many clusters.

DataStax has released version 2.0 of its Python driver for Apache Cassandra and DataStax Enterprise, featuring new capabilities such as query paging, lightweight transactions, and support for executing multiple prepared statements in a single batch. The driver is compatible with Apache Cassandra 1.2 and 2.0, as well as DataStax Enterprise 4.0, 3.2, and 3.1. It can be downloaded from PyPI or accessed on GitHub.

The text discusses strategies for loading data into Titan, a distributed graph database. It outlines different approaches based on the size of the data, ranging from millions to billions of edges. For smaller datasets (up to tens of millions of edges), common Gremlin operations and scripts executed through the Gremlin REPL are recommended. Larger datasets may require more advanced techniques such as BatchGraph for handling intermediate commits and maintaining a vertex cache. The text also provides examples using real-world datasets like Wikipedia Vote Network and DocGraph data set to demonstrate these strategies.

In the Test Engineering department at DataStax, Python is frequently used for test infrastructure, especially in clustered environments. To access shared cluster metadata directly from Python, a single Python metadata object can be created at a central location and easily accessed from any node of the cluster using Pyro4. This allows Python code running on one machine to call native Python code on another machine. By installing Pyro4 on each node of the cluster and setting up a nameserver and metadata server, remote Python objects can be used across the nodes. The Metadata class is used to create a metadata object that can be accessed by other nodes in the cluster after being registered with the nameserver. This method provides fast setup, easy use, simple maintenance, and straightforward addition of new operations.

Apache Cassandra 2.1 introduces a new distributed counters implementation that is safer, simpler, and often faster than previous versions. The crucial difference is that instead of logging counter deltas directly to the commit log, Cassandra now reads the current value for every counter update and applies the delta. This results in more accurate values and more consistent performance. However, increased latency may be experienced for RF=1/CL.ONE/replicate_on_write=false counter tables. The new implementation also creates fewer objects for the JVM to garbage-collect, reducing pathological behavior under heavy load. Counters are split into fragments called 'shards', and a counter's value is calculated as the sum of all shards' values in its replica set. In Cassandra 2.1, local shards are eliminated, and a lock is acquired for each counter being updated to read the current value before writing the incremented value. This simplifies almost every counters aspect and reduces garbage collection activity. A new form of cache called counter cache is introduced in Cassandra 2.1 to keep hot counter values performant, with configurable size and save period in cassandra.yaml. In Cassandra 3.0, the legacy local/remote shards will be removed entirely, eliminating the need for the counter cache.

Cassandra performs optimally when the required data is already in memory as disk operations are relatively slow. To design an effective data model in Cassandra, it's crucial to consider best practices such as writing rows to disk in the same order they will be read and utilizing PRIMARY KEY for ordering. In the example provided, a table designed for holding time-series status updates is created with a carefully designed primary key that ensures rows are stored on disk in reverse chronological order according to the status_id. This enables efficient retrieval of the last 10 status updates for a user. Cassandra's row caching ability can be utilized by enabling it and specifying the number of rows to cache per partition. To use the row cache, you must also instruct Cassandra how much memory you wish to dedicate to the cache using the row_cache_size_in_mb setting in the cassandra.yaml config file. To test if data is truly being retrieved from the cache rather than from disk, tracing can be enabled in cqlsh. The trace will indicate whether a disk read was necessary or not. If the cache is insufficient to complete the request, a disk read may be necessary, which can be mitigated by increasing the cache size limit or restructuring the table to place frequently accessed rows at the head of the partition. By studying your application's query model and tuning it according to these best practices, you can achieve great response times without needing an external caching layer.

The Cassandra Test Engineering team has been working to improve testing for the Cassandra upgrade process, with a focus on making sure nodes can use the latest releases' improvements without any negative impact on pre-upgrade data. They have made changes to their open-source python project, cassandra-dtest, which is run on a publicly accessible Jenkins server found at cassci.datastax.com. The team has simplified the test upgrade procedure, extended it to as many scenarios as practical, and minimized human intervention in keeping tests updated. They have also implemented a system that allows tests to automatically see new minor version releases and test them without any code changes needed.