July 2016 Summaries

Microsoft Azure is a popular cloud provider among enterprise companies, with over 66% of Fortune 500 companies using it. Datadog offers more than 650 integrations and now supports monitoring Azure App Service applications alongside other Azure resources such as SQL databases and VMs. Azure App Service is a platform-as-a-service for building web, mobile, and business applications, abstracting away the underlying infrastructure to allow developers to focus on code rather than maintenance. Datadog collects metrics from all four flavors of App Service: Web, API, Mobile, and Logic. With Azure App Service integration enabled in Datadog, users can monitor incoming requests, track HTTP response codes, and more. The integration also supports going agentless for monitoring applications running on multiple platforms.

Microsoft Azure is a popular cloud provider among enterprise companies, with over 66% of Fortune 500 companies using it. Datadog has integrated with Azure App Service, allowing users to monitor their web, mobile, and business applications alongside their Azure SQL Databases and other hosts, applications, and services. The integration provides out-of-the-box dashboards for monitoring incoming requests, HTTP response codes, and other metrics. Datadog's agentless approach to monitoring makes it easier to determine the root cause of application failures and errors by correlating data from multiple platforms, including Azure, on-premises, or elsewhere. With this integration, users can also track custom metrics specific to their application and visualize performance across arbitrary granularity using tags. To get started with Datadog's Azure App Service integration, users should first integrate Azure with Datadog and then enable the integration, adding it to their list of monitored technologies.

Lighttpd is an open source web server known for its flexibility and speed, providing a low memory footprint and small CPU load while remaining secure and compliant with web standards. Monitoring Lighttpd's performance can be challenging but features like its small CPU load make it easy to establish baselines, allowing users to ensure optimal performance. Datadog integrates with Lighttpd to help monitor key metrics such as CPU load, memory usage, and uptime, providing a visual representation of performance in a screenboard and enabling alerts for issues before they affect users. The integration also offers an out-of-the-box dashboard featuring useful metrics like number of requests, network traffic, and difference between hits/s and requests/s, helping to identify performance issues quickly. Additionally, the Datadog Agent collects various metrics from Lighttpd servers, including CPU usage, idle threads, uptime, bytes read and written, and request rates, allowing users to set up custom alerts for unexpected changes in these metrics. Overall, integrating Datadog with Lighttpd can help ensure optimal performance, security, and compliance of the web server.

Web application security has been stagnant for over a decade due to lack of changes in development processes. However, recent developments such as agile and Devops models, cloud infrastructure, and microservices have necessitated an update in web application security initiatives. Signal Sciences' integration with Datadog brings intelligent alerting engine for web application security into common practice. The Next Generation Firewall (NGWAF) by Signal Sciences incorporates visibility into real-time attack data, advanced detection and blocking techniques, and quick reaction to attempted and successful attacks. This integration provides an excellent platform for understanding collected security environment data, creating alerts on detected attacks, and correlating threat activities with their effects throughout the stack. It also improves security-data analytics and alerting by allowing correlation of multiple data sources including development, operations, and security integration points. Effective security relies on data measurement, which is provided by Datadog and Signal Sciences through surfacing and alerting on important metrics and data points.

Web application security has been stagnant for over a decade, with traditional development processes and tools no longer meeting the changing needs of modern applications. The integration of Datadog and Signal Sciences brings an intelligent alerting engine for web application security into common practice, providing visibility, blocking, and remediation capabilities to organizations. This collaboration enables the correlation of multiple data sources, improving security-data analytics and alerting, and allowing organizations to make informed decisions with measurable data. Effective security relies on data measurement, and joint customers such as AirBnB, Under Armour, Vimeo, Etsy, Taser, and others have seen significant improvements in their security levels while informing and improving their overall security processes.

IMMUNIO is a real-time security solution that detects and defends against web application attacks, providing protection from account takeover attacks, code vulnerabilities, and other threats. It integrates with Datadog to provide real-time security intelligence, enabling organizations to gain critical insights into how attacks are affecting their systems. With IMMUNIO, developers can identify abnormal activity, determine whether it's malicious, and take immediate action to stop attacks. The solution provides a comprehensive view of identified attacks, allowing teams to release apps into production secure in the knowledge that they will be immune to exploitation. It addresses application security from the inside, monitoring to detect application layer attacks and other vulnerabilities, and immediately blocking unwanted activity in real time.

This guide provides an overview of various methods to access and collect metrics from Hadoop components such as HDFS, MapReduce, YARN, and ZooKeeper. It covers collecting metrics via APIs and JMX for NameNodes and DataNodes, obtaining MapReduce counters, using Apache Ambari and Cloudera Manager for monitoring purposes, and gathering ZooKeeper metrics through Jmxterm or the four-letter words method. Additionally, it highlights Datadog's integrations with Hadoop components for comprehensive monitoring solutions.

This blog post delves into the key metrics that should be monitored to ensure optimal health and performance of a Hadoop cluster. It covers service-oriented monitoring, HDFS metrics, MapReduce counters, YARN metrics, and ZooKeeper metrics. The article provides detailed descriptions of each metric along with recommendations on when to alert based on these metrics. Monitoring these key metrics can help identify potential issues in a timely manner and ensure smooth operation of the Hadoop cluster.

Microsoft Azure is a popular cloud provider among enterprise companies and its adoption is growing rapidly. Datadog now allows users to monitor Azure SQL Database performance and usage alongside other Azure VMs and hosts, making it easier to manage infrastructure across platforms. Azure SQL Database is a hosted relational database service on the Azure cloud platform that offers automatic analysis of design and usage patterns for performance improvements. Key metrics include utilization, client connections, resource consumption, and deadlock counts. Datadog captures these metrics as well as those related to the unique measurement unit for Azure databases: Database Transaction Units (DTUs). The Azure Resource Manager deployment model allows users to organize cloud services into cohesive groups for easier management and billing isolation. Datadog accommodates this model by automatically tagging ARM metrics by resource group, allowing users to filter and aggregate metrics at that level.

Hadoop's architecture and subcomponents were previously introduced in a guide to monitoring Hadoop health and performance. This post delves deeper into each technology, exploring the key metrics exposed by Hadoop that should be kept an eye on for optimal cluster operation. The importance of treating DataNodes and NodeManagers like cattle was highlighted, as well as the need for a different mindset when monitoring HDFS compared to other systems. Key metrics include CapacityRemaining, MissingBlocks, VolumeFailuresTotal, NumDeadDataNodes, FilesTotal, TotalLoad, BlockCapacity/BlocksTotal, UnderReplicatedBlocks, and NumStaleDataNodes. Monitoring these metrics can help ensure high cluster availability, prevent data loss, and optimize resource utilization. Additionally, monitoring MapReduce counters such as REDUCE_INPUT_RECORDS, SPILLED_RECORDS, and GC_TIME_MILLIS is essential for identifying performance issues and optimizing application execution. YARN metrics, including unhealthynodes, activeNodes, lostNodes, appsFailed, totalMB/allocatedMB, progress, containersFailed, zk_followers, zk_avg_latency, and zk_num_alive_connections, also provide valuable insights into the health and performance of the cluster. By monitoring these key metrics, cluster administrators can take proactive steps to prevent issues, optimize resource utilization, and ensure high availability.

This post explores various ways to access metrics from core Hadoop components such as HDFS, MapReduce, and YARN, as well as from ZooKeeper. It discusses collecting metrics using standard development tools like JMX and API calls, as well as specialized tools like Apache Ambari and Cloudera Manager. The post also covers collecting MapReduce counters, Hadoop YARN metrics, and ZooKeeper metrics using both JMX and the four-letter words. Finally, it highlights the importance of having a comprehensive monitoring system that ingests performance metrics from multiple technologies in the data-processing stack.

Microsoft Azure is the second-largest cloud provider in the world and is especially popular among enterprise companies, with rapid growth in its adoption despite having a smaller market share compared to Amazon Web Services. Datadog provides a simple way to monitor all components of an infrastructure across various platforms, including Azure SQL Database performance and usage alongside other hosts, apps, and services. Azure SQL Database is a hosted relational database service on the Azure cloud platform, designed to be scalable and easy to manage, with automatic analysis of design and usage patterns for recommendations on performance improvements. Datadog's integration with Azure SQL Database pulls in key metrics such as utilization, client connections, resource consumption, and troubleshooting metrics like deadlocks, also capturing unique measurement units for Azure databases known as DTUs. Each Azure SQL Database is assigned a certain number of DTUs based on the chosen service tier and performance level, allowing users to ensure their database is properly sized to handle workload and identify when it's time to scale up. The Azure Resource Manager deployment model allows organizing cloud services into cohesive groups, making it easier to deploy and manage resources consistently, while Datadog accommodates this model via tags, enabling easy filtering and aggregation of metrics at the group level.

Hadoop is a framework for distributed computation and storage of large data sets on computer clusters, developed by Apache Hadoop. It consists of three core components: HDFS (Hadoop Distributed File System), MapReduce, and YARN (Yet Another Resource Negotiator). HDFS provides scalable and fault-tolerant data storage, while MapReduce is a framework for processing large datasets in a distributed fashion across multiple machines. YARN manages the allocation of computational resources for MapReduce jobs. ZooKeeper is used for coordination and synchronization of distributed systems, enabling high-availability of HDFS and YARN components. Understanding how these technologies work together is crucial for monitoring Hadoop health and performance.

The final installment of a series on monitoring Hadoop health and performance focuses on setting up detailed monitoring by integrating Datadog with Hadoop, including ZooKeeper. This guide explains how to verify the operational status of Hadoop and ZooKeeper components before installing the Datadog Agent, which collects and reports metrics for visualization and alerting. Detailed instructions are provided for configuring the Agent for various Hadoop components like NameNode, DataNodes, YARN, and ZooKeeper, using template configuration files. Once set up, users can monitor Hadoop's performance through a comprehensive dashboard in Datadog, alongside metrics from other systems, and set automated alerts for potential issues. The article encourages users to try a free trial of Datadog for enhanced visibility and alert capabilities in their data-processing infrastructure, with the source Markdown available on GitHub for further reference.

This article discusses three anti-patterns in visualizing monitoring data and suggests better solutions. Firstly, the author discourages using stacked area graphs for large groups as they become less effective with increasing size of the group, leading to loss of vertical resolution. Instead, it recommends using an aggregate timeseries graph for the big picture and complementing it with focused looks at individual behavior through heatmaps, host maps, line graphs, and outlier detection graphs. Secondly, the author advises against using one tool for everything and suggests using heatmaps and bar graphs instead of line graphs in certain cases. Lastly, the article warns against summing something that shouldn't be summed and recommends using line graphs or heatmaps depending on whether precision or trends are more important. The key takeaway is to use appropriate graph types for different metrics and scale levels to ensure clarity and readability of monitoring data visualizations.

Datadog is a powerful monitoring tool that allows users to collect, aggregate, and graph performance metrics from various parts of their infrastructure such as servers, databases, containers, and cache engines. It also enables users to monitor almost anything by writing Agent Checks or running HTTP checks. A unique example of this versatility is the creation of a website called www.IsPokemonGODownOrNot.com that uses Datadog's features to check server availability and login functionality for Pokémon GO in both Europe and the U.S. This dashboard helps users determine if the game is down or experiencing issues, allowing them to stay informed about their gaming experience.

This post highlights common pitfalls in graphing monitoring data and provides guidance on how to avoid them. It discusses the use of phyllo graphs, which are often misused due to their inability to handle large groups effectively, leading to visual overload. The solution is to use aggregate timeseries graphs for big-picture views, complemented by focused graphs such as heatmaps, host maps, and line graphs that track performance envelopes or outlier detection. Additionally, the post cautions against using one tool for everything, as this can lead to "spaghettification" of data, and instead recommends using bar graphs for sparse metrics, aggregating metrics when precision matters most, and employing heatmaps when trends are paramount. The key takeaway is that careful consideration of graph types and design can improve the clarity and usability of monitoring dashboards.

Datadog is a monitoring tool that allows users to collect, aggregate, and graph performance metrics from any part of their stack. It can be used to monitor anything, including custom data sources, websites, and applications. Datadog's powerful features enable the creation of custom dashboards and checks to measure server availability, latency, login availability, and other metrics. In this example, a team created a dashboard to monitor Pokémon GO's availability, using HTTP checks against Niantic's data endpoint to calculate server availability and latency, as well as logging in to the game every 30 seconds to measure login availability. This allows users to quickly check if the game is down or not, and provides a customized solution for monitoring infrastructure and applications.

Apache CouchDB is an open-source NoSQL database designed for web applications, using a document-oriented approach with data stored in JSON and accessible via HTTP requests. It supports complex indexing and view updates through JavaScript, and its asynchronous replication ensures eventual consistency, making it suitable for scalable, public-facing apps. Datadog enhances CouchDB monitoring by providing real-time and historical metrics on traffic, latency, and workload, helping identify issues like high delete request rates or the need for database compaction due to excess old document revisions. Additionally, Datadog tracks the authentication cache to detect potential security threats, such as unauthorized access attempts, by monitoring cache misses and unauthorized request metrics. For those using Datadog, integrating CouchDB monitoring is straightforward and offers insights alongside over 850 other technologies.

At GopherCon US 2016, cgo bindings for two compression libraries, czlib and zstd, were released. These libraries have been used in production at Datadog. The performance of these libraries varies significantly depending on the interface used. Benchmark results show that using a non-streaming interface in czlib can be faster than the standard library's compress/zlib for both compression and decompression. Zstd, developed by Yann Collet, is a fast compression library with similar performance to zlib but with much faster decompression times. The zstd Go binding mimics the zlib interface and supports advanced features like stream compression, compression levels, and pre-computed dictionaries.

Jason Moiron from Datadog is releasing two new cgo bindings for compression libraries czlib and zstd, which have been used in production by Datadog. The goal was to improve performance of their data pipeline using zlib compressed messages. czlib is a modified version of the vitess project's cgzip package that uses zlib wrapping instead of gzip headers, resulting in faster encoding and decoding speeds. In contrast, zstd is a relatively new fast compression library from Yann Collet, which compresses slightly faster than zlib at level 6 and decompresses much faster, making it a great general-purpose zlib replacement. The zstd Go binding exposes common interfaces like stream compression, compression levels, and pre-computed dictionaries, making it functionally similar to the zlib interface with some additional features.

AWS Trusted Advisor is a new feature in the AWS Management Console that provides best practice recommendations for optimizing cost, performance, security, and fault tolerance through four core checks. The Service Limits Check gives granular resource consumption data across different AWS services. Integrating Trusted Advisor with Datadog allows users to monitor these metrics on an out-of-the-box dashboard, enabling quicker optimization of resources and infrastructure scaling. With this integration, users can set up custom monitors and alerts for unexpected changes in resource consumption, providing more precise control over service resource usage. Setting up Trusted Advisor with Datadog is simple and provides valuable insights into AWS performance and usage.

AWS Trusted Advisor is a new component introduced into the AWS Management Console that analyzes usage across services and provides best practice recommendations for optimization. It offers four core checks to help customers optimize their cost, performance, security, and fault tolerance. The integration of AWS Trusted Advisor with Datadog allows users to monitor resource consumption data in real-time, providing actionable insight into their AWS infrastructure. This integration enables users to quickly see how custom optimization recommendations from Trusted Advisor affect their resource allocation and consumption, making it easier to scale their AWS environment. With this setup, users can set up custom monitors and alerts within Datadog to gain more precise control over their services' resource consumption, ultimately improving their AWS performance and usage management.

Amazon OpsWorks is a service designed to streamline the management of applications hosted on AWS by utilizing the Opscode Chef framework to automate and customize application configurations, updates, and deployments in cloud environments. It organizes infrastructure into three tiers: stacks, layers, and instances, allowing for efficient resource management and scalability. Datadog integration enhances this by enabling monitoring of OpsWorks events and metrics, such as CPU, load, memory, and processes, providing insights into resource utilization and performance. Users can filter or aggregate metrics by stack, layer, or instance, and set thresholds for load-based instances to optimize resource allocation. By integrating with Datadog, users can correlate event data and metrics, ensuring their recipes execute as intended and aiding in strategy formulation for managing scalable applications. Setup is straightforward for existing AWS integrations in Datadog, with options to incorporate AWS CloudTrail for event tracking.