October 2017 Summaries

Amazon Web Services (AWS) Elastic Compute Cloud (EC2) allows easy launch and termination of virtual machines, while AWS Auto Scaling takes it further by automating the process. Datadog's Auto Scaling integration enables tracking metrics and events from Auto Scaling groups alongside other AWS services. The integration provides an out-of-the-box screenboard that displays recent Auto Scaling events, shows changes in Auto Scaling groups over time, and indicates their size distribution. AWS Auto Scaling responds to changes in demand for EC2 resources by adjusting the number of instances accordingly. It can be configured based on specific application needs, such as regular scaling schedules, dynamic scaling policies, or manual scaling with minimum and maximum limits. Datadog's integration helps determine appropriate configurations for Auto Scaling groups as infrastructure grows. Users can find key demand metrics by graphing, comparing, and correlating resource metrics across Auto Scaling groups in Datadog. Each Auto Scaling metric is automatically tagged with its autoscaling_group, allowing targeted dashboards to be created. By tracking resource usage over time, users can select an appropriate metric for each group's scaling policy. Datadog also helps monitor the health of Auto Scaling groups and their responses to changes in demand. Users can set alerts to gauge the health of their groups and determine if they respond appropriately when demand fluctuates. Additionally, Datadog enables users to view events from any Auto Scaling group or focus on a single group using the autoscaling_group tag for filtering queries. To monitor AWS Auto Scaling with Datadog, enable Auto Scaling in the AWS integration tile after setting up the Amazon Web Services integration. A 14-day free trial is available for new users.

AWS Elastic Compute Cloud (EC2) provides automatic scaling capabilities through AWS Auto Scaling, which adjusts the number of virtual machines based on demand. Datadog's integration with AWS Auto Scaling allows users to track metrics and events from their Auto Scaling groups in one place, providing insights into performance and availability. The integration includes an out-of-the-box screenboard that displays recent events and shows how group sizes have changed over time. With this integration, users can configure scaling policies based on specific needs, including dynamic scaling, manual scaling, and regular scaling schedules. Datadog's integration also helps determine the most appropriate ways to configure Auto Scaling groups to ensure infrastructure stays in step with demand. Additionally, users can track key demand metrics, set alerts for health checks, and monitor AWS Auto Scaling with Datadog, all within a single platform.

Algorithmic monitoring, such as Datadog's outlier detection and anomaly detection, uses machine learning to automatically identify abnormal values in user traffic, critical business metrics with recurring fluctuations, and deviations from normal group behavior. These features can help detect issues in infrastructure and applications more effectively than static thresholds or rate-of-change alerts, reducing false positives. By combining anomaly detection and outlier detection, users can gain more comprehensive insights into their systems' performance.

Datadog's algorithmic monitoring capabilities use machine learning functionality to automatically identify abnormal values in metrics, based on analyses of group behavior or past performance. This allows for the detection of issues such as gradual baseline shifts or recurring fluctuations, which are difficult to catch with traditional threshold-based alerts. Algorithmic monitoring can be used to uncover abnormalities in user traffic, periodic fluctuations over changing baselines, and abnormal loads in distributed databases. By combining anomaly detection and outlier detection, users can gain more fine-grained insights into their infrastructure and applications. This enables the delivery of smarter alerts for issues such as dips in user traffic during peak business hours, or imbalances in load distribution across web servers.

GovPredict, a platform providing unique data and insights to clients, has transitioned from manual monitoring to using Datadog for improved problem diagnosis, system performance optimization, ensuring data quality, and reducing developer burden of maintaining a monitoring stack. By integrating Datadog into their systems, they have significantly reduced the time taken to diagnose data quality issues and increased overall data consistency. Additionally, Datadog has been helpful in diagnosing downtime and optimizing database loads for production applications and databases.

GovPredict, a platform providing unique data and insights to enterprise and government clients, faced challenges with manual monitoring of its complex data sources. The company transitioned to using Datadog for data quality, diagnosing issues in minutes rather than hours, and optimizing production applications and databases. By leveraging Datadog's features, GovPredict was able to reduce errors, improve data consistency, and enhance overall system performance, ultimately increasing the efficiency of its developers and improving the accuracy of its data-driven insights.

The Datadog user community recently convened in Austin for their annual summit, where they discussed new features and shared experiences with monitoring and observability. Among the highlights were the introduction of Agent 6.0, a comprehensive rewrite of the Datadog Agent; advances in container monitoring; and insights into how users can leverage logs to troubleshoot complex problems. Additionally, attendees heard from customers like Caviar about their transition from monolithic applications to microservices with the help of Datadog APM. The summit also featured talks on algorithmic alerting and machine learning for predictive alerting.

The Datadog user community gathered in Austin on September 28 for a Summit event, where they learned about new features, heard from peers and the team, and exchanged ideas. The newly released Agent version 6.0 is a complete rewrite written in Go, offering improved performance with multi-threading and resource control, while still supporting Python checks. Datadog's log analytics capabilities were also showcased, explaining how logs complement the platform and can be used to troubleshoot complex problems. Additionally, new tools for monitoring containers, advances in container monitoring, and algorithmic alerting techniques using machine learning algorithms were demonstrated, highlighting various ways customers are utilizing Datadog's features.

Error monitoring and tracking are crucial in software development to identify issues and fix bugs. Sentry is an event logging platform that automates error monitoring, recognition, and tracking by capturing and aggregating exceptions. It notifies developers of exceptional events and updates its dashboard with events and exception stack traces. Integrating Sentry with Datadog allows teams to search and comment on errors, correlate errors with metrics from other systems, and create custom dashboards for better understanding of error patterns. Setting up the integration is simple and can improve debugging efficiency.

Elastic Load Balancing (ELB) by Amazon Web Services (AWS) directs traffic to multiple targets, distributing workload evenly among them. It includes Classic Load Balancers, which route traffic to a backend pool of EC2 instances across various availability zones, and Application Load Balancers (ALB), which enhance functionality by allowing routing to multiple ports on the same instance and supporting path-based routing. Monitoring ELB is crucial as it serves as the primary gateway for applications; unexpected issues can lead to overwhelmed targets or application downtime. By integrating with Datadog, users can monitor essential metrics like latency, error codes, and request throughput, and access ALB dashboards to track healthy and unhealthy hosts. Correlating these metrics with other infrastructure data can help diagnose issues, and anomaly detection features can alert users to unusual traffic patterns. Additionally, monitoring ALB latency and response types can provide insights into backend performance, with Datadog's application performance monitoring offering detailed request tracing to identify sources of latency.

The article delves into four types of status checks used in monitoring and alerting systems, specifically host, service, process, and network checks, which are designed to assess the up-or-down status of various components within an infrastructure. Host checks alert administrators when a monitoring agent on a host stops sending signals, while service checks fire alerts if a service fails to connect after consecutive attempts. Process checks, similar to service checks but at a lower level, monitor the status of specific processes, especially useful for custom-built services, and network checks assess the connectivity between locations and endpoints, identifying regional issues. These checks can be applied individually or at a cluster level, where the latter is often preferred to mitigate alert fatigue by alerting on widespread issues rather than isolated incidents. The discussion highlights the importance of actionable alerts, particularly for critical, non-redundant services or widespread failures, ensuring responders have clear remediation steps. The subsequent installment promises to explore more continuous monitoring approaches through timeseries metric evaluations.

In the article, different types of alerts for monitoring infrastructure metrics and events are explored, emphasizing the importance of understanding each alert's ideal use case. Threshold alerts are used to trigger notifications when metrics exceed a set value over a specified time, while change alerts track significant variations in metrics with variable baselines. Outlier alerts identify deviations from expected group behavior, useful for distributed systems, and anomaly alerts detect deviations from historical trends, accommodating patterns like seasonality. Event alerts focus on discrete occurrences, such as failed jobs, and composite alerts combine multiple conditions to address complex issues. By leveraging these alert types, users can promptly address critical issues in their environment.