November 2013 Summaries

At AWS re:Invent, it was highlighted that assessing the health of Amazon infrastructure is challenging, particularly in fluctuating demand environments with auto-scaling deployments. In response to this feedback, Datadog has introduced a new AWS Overview Screenboard, which provides an instant overview of the entire AWS infrastructure and can be customized intuitively. The screenboard monitors major components of the environment and allows users to identify hot spots and services requiring attention. It also enables users to modify metrics and customize views for different teams or users working off the same AWS environment.

The new AWS Overview Screenboard provides a concise and visual way to assess the health of entire Amazon infrastructure, helping users quickly identify hot spots and areas requiring attention. This screenboard offers a high-level overview of major components, allowing users to "slice and dice" metrics by availability zone or per ELB using template variables. The screenboard can be customized with widgets, adding, modifying, changing colors, sizes, and annotations, enabling users to tailor views for different teams or users working off the same AWS environment. With a free trial available, users can gain access to this powerful tool in just a few minutes.

Datadog's Matt Perpick delivered a presentation titled "Building High-Volume Data Systems in the Python Ecosystem" at PyData 2013 in New York, where he discussed constructing high-throughput data pipelines using the Python, Numpy, and Cython stack employed at Datadog. The talk emphasized methods for enhancing data processing capabilities within the Python ecosystem and showcased Datadog's approach to managing large volumes of data efficiently. Attendees interested in understanding the practical applications of these high-volume data systems were encouraged to explore Datadog's platform through a free trial.

AWS Elastic Load Balancers (ELB) have been lacking in monitoring metrics since their introduction in 2009. However, with the recent addition of three new metrics - BackendConnectionErrors, SurgeQueueLength, and SpilloverCount - ELB monitoring has become more comprehensive. Essential ELB monitoring metrics include HealthyHostCount, Latency, HTTPCode_ELB_5XX, SurgeQueueLength, and SpillOverCount. Other secondary metrics are RequestCount, HTTPCode_ELB_4XX, and HTTPCode_backend_yXX. Composite ELB metrics can be computed by combining multiple metrics to better understand load balancer behavior. AWS CloudWatch metrics have a timeframe of 60 seconds, which is important to consider when monitoring counts per minute.

AWS Elastic Load Balancers (ELB) offer improved monitoring capabilities with the introduction of three new metrics: BackendConnectionErrors, SurgeQueueLength, and SpilloverCount. These metrics provide insights into the load balancer's performance, such as connection errors, queue length, and spill-over requests. To effectively monitor ELB, it is essential to understand how these metrics interact and use them in conjunction with other metrics like RequestCount, HTTPCode_ELB_5XX, HealthyHostCount, Latency, and HTTPCode_backend_yXX. The new metrics should be used to identify issues such as too many inbound requests, latency, or server misbehavior, allowing for prompt action to be taken to improve the load balancer's performance and prevent frustrated customers.