October 2013 Summaries

Today at the Cassandra Summit in London, DataStax has announced the general availability of DataStax DevCenter 1.0. This new tool is designed for developers and administrators to create a simple, productive, and familiar environment for working with Apache Cassandra™ and DataStax Enterprise. DevCenter is a free visual query tool that supports CQL queries and commands against both platforms. Since its pre-production release, several core components have been added, including a Schema Explorer, an Outline view for CQL scripts, and a smart CQL editor with syntax highlighting and advanced code auto-completion. DataStax DevCenter is available for download on Mac, Linux, and Windows platforms. The distribution includes sample scripts to help users get started. Feedback and suggestions can be sent to [email protected].

The text discusses important operational aspects of running Apache Cassandra, an open-source distributed database management system. It covers the following key points: 1. Repair: This process ensures data consistency across replicas but can be expensive in terms of resources and latency. Running repair weekly is recommended. 2. Read_repair_chance: This setting controls how often Cassandra checks for inconsistencies between replicas during reads. The default value is 0.1, which means that 10% of requests will trigger a background read repair. 3. Cleanup: This process removes data no longer owned by a node after topology changes. It's recommended to schedule cleanup only when necessary and not at regular intervals. 4. Compaction: This optimization process merges rows in the background to reduce IO and CPU time for reads. Major compactions can exacerbate issues with tombstones and updates, so it's better to let compaction run its natural course. 5. JVM Heap Size: For optimal performance, Cassandra requires a small heap size (ideally less than 12 GB). Memory not allocated to the heap is utilized by Cassandra for memory-mapped IO, improving overall efficiency.

This blog post discusses how to optimize Hadoop configuration settings for better performance with DataStax Enterprise (DSE) and Apache Cassandra®. It covers basic settings affecting the operating system, JVM memory, and disk, as well as specific parameters in core-site.xml and mapred-site.xml files that can improve DSE Hadoop Map/Reduce job performance. The post also highlights how DSE 3.1 and later versions provide auto-tuned default values for some configuration settings to help users determine optimal values.

Rapid Read Protection (RRP) in Cassandra allows the system to tolerate node failure without dropping a single request. It was designed for version 2.0 and will be included in the upcoming release of Cassandra 2.0.2. RRP monitors outstanding requests and sends redundant requests to other replicas when the original is slower than expected, improving system resilience and reducing latency variance. By default, it uses a 99th percentile setting for retry configuration, which balances between not performing too many extra requests while still addressing worst-case problems.

Cassandra File System (CFS) is a HDFS-compatible file system implemented on top of Cassandra. It is fully distributed and has no Single Point of Failure (SPOF). CFS includes two tables - inode and sblocks, which store metadata and data blocks respectively. When using CFS, you may occasionally encounter problems such as orphan blocks and lost blocks. Orphan blocks occur when a file stream is not closed properly, leaving unreferenced data blocks in the sblocks table. To remove these orphan blocks, use the dsetool repaircfs command. Lost blocks problem occurs when there exists an inode of the file but one or more referenced data blocks cannot be read. This can be caused by insufficient Consistency Level for writes into CFS or corruption of CFS data files. To diagnose CFS inconsistencies, use the dsetool checkcfs tool, which has two modes of operation: recursive checking directories and checking single files. Running nodetool repair cfs can fix some issues, but permanent file corruption may require deleting and re-saving the file. Using a Replication Factor of at least 3 and Consistency Level at least CL.QUORUM is recommended to avoid these problems.