February 2016 Summaries

The Google Cloud Platform experienced a major outage on February 8th, 2016, affecting the Redis Cloud service. The outage was caused by a software bug introduced during a weekly upgrade to the service's software. The issue led to multiple nodes in one cluster failing simultaneously, causing downtime and manual recovery efforts. The company apologized for the impact on its customers and is taking immediate steps to prevent similar incidents, including suspending automatic weekly upgrades, revisiting DevOps code deployments, and improving tooling to handle DNS request throttling.

The spark-redis package provides a Redis connector for Apache Spark, allowing users to access and manipulate Redis data structures as RDDs. To test its capabilities, the author set up a standalone Apache Spark cluster and used it to count words in Redis's source code files. The results were then stored in a Sorted Set for later querying. The author also demonstrated how to store word counts for each file in its own Sorted Set and reduce them to a single total using the spark-redis connector. Additionally, the package provides cluster-aware data access and reduces inter-engine shuffling, making it a powerful tool for big data processing.