October 2014 Summaries

The DataStax Java driver for Cassandra uses an asynchronous architecture, allowing client code to get query results in a non-blocking way via Future instances. This post explores the concept and demonstrates its use in implementing a client-side equivalent to the SELECT...IN query. It also discusses how to handle timeouts and exceptions using ResultSetFutures, which extend ListenableFuture from Google's Guava library. The post further explains how to update a hypothetical GUI with the result of a query using callbacks. Finally, it presents an efficient approach for retrieving data from various partitions in Cassandra by sending individual queries and collating the results client-side.

The blog post provides guidance for troubleshooting common issues encountered by new Apache Spark users, particularly those using DataStax Enterprise Spark Standalone. It addresses the typical initial error of a Spark application requesting more resources than available, highlighting the importance of managing cores and RAM through Spark configuration settings, such as `spark.deploy.defaultCores` and `spark.cores.max`. It discusses the issue of OutOfMemory (OOM) errors due to excessive data caching and recommends configuring the `spark.cleaner.ttl` to manage memory usage effectively. The text also touches on class not found errors, advising the inclusion of all dependencies in a fat JAR or ensuring consistency in library versions across the cluster. Additionally, the post explains how to navigate the Spark UI to monitor resource usage and application statuses, offering insights into tasks, stages, and RDD storage for performance debugging. These troubleshooting tips aim to help users optimize Spark applications and efficiently manage cluster resources.

DataStax Community Edition versions 2.1.1 and 2.0.11 are now available in public repositories for download. These maintenance releases include significant changelog entries, which were made possible by valuable bug reports from the Apache Cassandra JIRA user community. The contributions of these users have helped improve the overall quality of DataStax Community Edition.

Proper error handling in Cassandra is crucial to ensure data safety and continuous availability of clusters. Understanding the various types of exceptions thrown by drivers can help developers handle errors effectively. Some common exceptions include NoHostAvailableException, UnavailableException, ReadTimeoutException, WriteTimeoutException, QueryValidationException, and others. Handling these exceptions properly involves understanding their causes and implementing appropriate retry policies. Using idempotent operations, such as CAS (Compare-And-Set) operations, can also help manage errors in Cassandra. Additionally, developers should be cautious when using non-idempotent operations like counter updates or list append/prepend operations, as they may lead to data inconsistencies.

Instagram's Infrastructure Software Engineer, Rick Branson, explains how implementing Apache Cassandra significantly reduced costs and improved scalability for their backend systems. Initially using Redis for storing auditing information related to security and site integrity, they switched to Cassandra due to rapidly growing data size and high write rate with low read rate. The transition resulted in a 75% cost reduction and allowed them to add nodes as needed. They also ported their "inboxes" or newsfeed part of the app to Cassandra, which was previously in Redis and faced memory limitations. This move improved durability and reduced costs while maintaining good performance. Branson recommends thoroughly reading Cassandra documentation for successful implementation.

The latest version of the DataStax Java driver introduces support for version 3 of the native CQL protocol. This new version models protocol versions as a Java enum and increases the number of stream ids per connection from 128 to 32,768. It also allows setting default timestamps with each query and supports serial consistency levels in lightweight transactions on BatchStatements. Additionally, it includes other improvements and fixes.