August 2014 Summaries

An update has been released to improve monitoring of Amazon SQS, providing better visibility into the health of message queues, their traffic patterns, and the state of messages. This enables users to identify and fix performance issues more effectively. Amazon's Simple Queue Service (SQS) is a scalable, managed message queue in the AWS suite of services, designed for applications that stream analytics. Key SQS metrics include number_of_messages_sent, number_of_messages_received, number_of_messages_deleted, sent_message_size, approximate_number_of_messages_visible, approximate_number_of_messages_delayed, and approximate_number_of_messages_not_visible. Datadog has created an out-of-the-box screenboard to help users monitor their SQS usage, allowing for visualization of queue performance alongside other AWS services and applications.

Amazon has released an update to its integration with Datadog, making it easier for users of the Simple Queue Service (SQS) to monitor their message queues and identify performance issues. SQS is a scalable, managed message queue in the AWS suite of services, well-suited for applications that stream analytics and decoupling services to avoid a domino effect. To properly use SQS, it's essential to ensure queues are not continuously increasing in length or going unused, which can lead to performance issues such as too many empty receives, messages deleted, and sent message size affecting the service cost. Datadog has created an out-of-the-box screenboard to help users stay on top of their SQS usage, providing immediate insights into message queues, including metrics such as number of messages received, processed, and delayed, as well as a side-by-side comparison of traffic flow and throughput. Users can sign up for a free 14-day Datadog trial to enable the AWS SQS integration and monitor performance in conjunction with their applications and other AWS services.

The Top List visualization in Datadog ranks infrastructure components by metric values, highlighting the most heavily loaded hosts, fullest disks, or pages with worst latency. This tool helps identify and solve problems within an infrastructure. It can be used to monitor specific application versions or customer types by creating custom tags. The ranking is based on metrics like load, memory use, or custom app metrics. Top Lists work well in screenboards for easily digestible information sharing and as a complement to timeseries or heatmap graphs.

Jonathan Gala from Datadog discusses the importance of monitoring infrastructure components, highlighting the benefits of using Top Lists to identify best- and worst-performing components. Top Lists provide a visual representation of metric values, outliers, and distribution shapes, allowing users to quickly identify problems or areas for improvement. The feature is particularly useful in identifying under-used database indices, reclaiming space and performance, and providing insights into customer infrastructures. With customizable tagging options and various visualization settings, Top Lists enable users to gain a deeper understanding of their infrastructure's performance and make data-driven decisions.

VMware vSphere is a server virtualization platform that enables organizations to manage and provision virtual machines at scale. Datadog's vSphere integration provides real-time metrics and events monitoring, allowing users to track dynamic environments and address resource bottlenecks. The integration offers various features such as customizable dashboards, forecasting algorithms, and fine-tuning of metric collection to optimize performance. It also monitors CPU utilization across clusters, memory ballooning, and datastore disk usage, providing 360-degree visibility into the vSphere environment. With its flexibility in collecting data from resources and metrics, Datadog's integration helps users quickly address performance issues and maintain a highly performant vSphere deployment.

Datadog has released a new version of its Agent (dd-agent 5.0.0), expanding OS support to include RedHat 7, CentOS 7 and the latest versions of Fedora. The update also adds an integration for collecting metrics from SNMP and expands Docker metric collection capabilities. This allows users to monitor large numbers of containers and visualize new Docker metrics such as running and stopped containers, and number of images on a ScreenBoard. The new Agent contains all libraries needed for agent-based integrations, eliminating the need to worry about dependencies. A free 14-day trial is available for those interested in accessing these new features.

The Datadog Agent, written in Python, faced challenges with its installation process due to varying versions of Python across different OS distributions and the need for third-party libraries. Before Agent 5.0, users had to manually install dependencies, which could lead to compatibility issues or conflicts with existing applications. To solve this problem, Datadog implemented Chef Omnibus, a tool that creates full-stack installers for projects across various platforms. The new build process combines Github, Jenkins, Vagrant, and S3 to create self-contained installer packages containing all the dependencies needed by the Agent. Upon installation, these dependencies are isolated in /opt/datadog-agent, ensuring compatibility with existing applications. This improvement allows users to add integrations and access new features more easily without worrying about dependency conflicts or issues related to different Python versions.

The Datadog Agent has been released with version 5.0.0, expanding OS support to include RedHat 7, CentOS 7, and the latest versions of Fedora, making configuration simpler for many integrations. The new release also adds SNMP metrics collection and expands Docker support, allowing users to monitor large numbers of containers and visualize key metrics on ScreenBoards. Additionally, the Agent contains all necessary libraries, eliminating dependency concerns, and a free trial is available for users who want to quickly collect, graph, analyze, and alert on the new metrics. The community's contributions were instrumental in making this release possible.

Datadog introduces the host summary panel as a new tool to simplify troubleshooting of problematic hosts. The panel provides an out-of-the-box view of all metrics collected by Datadog for each host, grouped by integration or application. This feature helps users identify patterns and correlations among host metrics when investigating ambiguous alerts like high system load. By scanning the host summary panel, users can gain insights into a node's role and behavior, as well as investigate potential causes of issues. The host summary panel is accessible from the Datadog Infrastructure page and offers panoramic infrastructure visualizations and rich alerting features for more targeted monitoring coverage.

Datadog has introduced the host summary panel, a new tool to simplify troubleshooting of ailing hosts. This panel provides panoramic views of every metric Datadog collects for each host, grouped by integration or application. By using this panel to investigate an alert, users can identify patterns and find correlations among host metrics, helping them to troubleshoot issues more efficiently. The host summary panel is particularly useful in cases where a general alert such as "System load is high on host:i-abcd1234" is received, allowing users to quickly scan through the host's metrics and gain insight into the underlying cause of the issue.

Datadog is proud to announce that it now supports deployment sizes ranging from a single machine to warehouse scale, making it easy to install and configure a monitoring solution for environments running CoreOS. The Datadog Agent can be easily installed on CoreOS in under 5 minutes, with the option to use Docker containers for quick testing. With its streamlined installation process and complete view into infrastructure, Datadog aims to ease the pain of monitoring at any scale. By using Datadog, users can monitor their entire infrastructure, including container metrics, data analytics, and application-level monitoring. A free 14-day trial is available to try out Datadog for CoreOS-based containers.

A new integration between Datadog and VictorOps has been announced, allowing users to receive Datadog alerts in their VictorOps timeline for critical server, database, and application events. The integration enables the creation of VictorOps alerts directly from the Datadog stream, with customizable notification recipients. Users can update alert statuses in VictorOps using tags from the Datadog Events Stream. Once notified of an alert, users can investigate its cause by correlating system metrics to events within Datadog. To benefit from this integration, sign up for a free trial of both Datadog and VictorOps.