June 2015 Summaries

Monitoring data collected from various systems can be categorized into work metrics and resource metrics. Work metrics capture the top-level health of a system by measuring its useful output, while resource metrics help reconstruct a detailed picture of a system's state. Key characteristics of good monitoring data include being well-understood, granular, tagged by scope, and long-lived. The collected data can be used to generate alerts, diagnose issues, and investigate problems. It is essential to instrument everything, collect as many relevant metrics and events as possible, and retain them at full granularity for a sufficient amount of time to maximize their value.

Automated alerts are crucial for effective monitoring of infrastructure, as they help identify and address issues promptly to minimize service disruptions. However, alerts can be ineffective if overwhelmed by noise, making it essential to implement a strategic alerting framework, emphasizing "alert liberally; page judiciously" and "page on symptoms rather than causes." Alerts should be categorized by urgency: low-severity alerts record data for future reference without requiring immediate attention, moderate-severity alerts notify relevant parties of potential issues that need timely intervention, and high-severity alerts, or pages, demand immediate action for critical problems impacting service. Focusing on symptoms ensures that alerts are durable and relevant, allowing for quick response to actual service issues without unnecessary disruptions. Additionally, early warning alerts for critical resource limits, like disk space, can prevent severe problems by allowing preemptive action. This approach aims to optimize monitoring efficiency, reduce alert fatigue, and maintain service quality.

Remi Hakim introduces the integration of Microsoft .NET with DogStatsD, a tool that enables the tracking of C# application performances by sending custom metrics to Datadog for real-time graphing and analysis. The new feature allows events to be sent directly from Microsoft .NET to DogStatsD, facilitating the correlation of custom events with application metrics, such as identifying if specific events coincide with metric spikes. To implement this, users need a Datadog account, the NuGet package manager, and a running Windows Datadog Agent. Installation involves using Visual Studio's Package Manager Console to execute a command to install the DogStatsD C# client, after which events can be sent from the application. Additional documentation is available on the library's GitHub repository.

The text discusses how Datadog supports Docker's Ecosystem Technology Partner (ETP) program and provides an overview of Docker performance monitoring using the Datadog Agent. It explains that the Agent runs in its own container on a host to monitor all other containers, utilizing native cgroup accounting metrics for CPU, memory, network, and I/O data collection. The use of tags is highlighted as crucial for tracking and monitoring multiple containers without additional effort. Tags are also useful for defining alerts spanning across clusters of containers. Additionally, the text mentions how to monitor a container's lifecycle using Docker events and provides an example of exploring Docker metrics in Datadog's Metrics Explorer.

Datadog has partnered with Docker to support its ecosystem through the ETP program. The Datadog Agent can run in its own container on a host and monitor all other containers that run on that host, using native cgroup accounting metrics to gather CPU, memory, network, and I/O metrics of the containers every 15 seconds. This allows users to track and monitor multiple containers effortlessly with tags, which are used to define the metrics shown in dashboards and graphs based on one or many tags. Tags are also useful for defining alerts that span clusters of containers, such as alerting when a Redis container is running out of memory. Additionally, Datadog provides features to monitor Docker containers' lifecycle, explore Docker metrics, and sign up for a free trial to get started with monitoring Docker in your environment.