May 2018 Summaries

Mia Henderson, site reliability engineer at PagerDuty, discussed how her team integrated automated metric checks into their deployment process during Datadog Summit San Francisco in 2017. The team initially incorporated a canary phase into their build pipeline to limit the risk of introducing bugs and performance issues into production. However, they later adopted continuous deployment, automating the process of deploying and monitoring the canary code. A pivotal incident led them to automate the process of evaluating their canary releases by creating scripts to check key metrics using Datadog's Ruby client library. PagerDuty now integrates automated canary checks into their build and deploy pipeline, checking metrics after five minutes of a canary release. Mia encourages others to adopt this discipline and integrate metric checks into continuous deployment processes.

The PagerDuty team integrated automated metric checks into their deployment process using Datadog, a cloud-based monitoring and analytics platform. The team initially used canary releases to limit the risk of introducing bugs and performance issues into production, but later adopted continuous deployment to automate the process. However, this shift led to a pivotal incident where a developer's change caused an issue that wasn't caught in time to halt the canary release. To mitigate this risk, the team automated the process of evaluating their canary releases by creating scripts to check key metrics using Datadog's Ruby client library. The team integrated these checks into their build and deploy pipeline and encouraged others to adopt a similar discipline by writing their own check scripts and integrating them into continuous deployment.

Chef is an automation platform used for configuration management of infrastructure, supporting continuous delivery workflows. It uses policies to manage workflows and operational requirements across machines based on their server type or development process stage. The three foundational elements of Chef are nodes, a workstation, and the Chef server. Nodes are any machines managed by Chef, while the workstation is used for creating configurations and the server acts as a hub for all infrastructure configurations. Chef enables efficient management of infrastructure through recipes, which are Ruby files containing elements needed to set up parts of a system. A collection of recipes is called a cookbook. Roles define what nodes should do along with how they should be configured via a run-list of recipes and associated configuration attributes. Datadog provides a cookbook that includes a dd-agent recipe for installing the Agent, along with other elements to set up common integrations. By using Chef roles, you can efficiently deploy the Datadog Agent on multiple nodes within your infrastructure, saving time from manual installation on each node. Additionally, Chef enables monitoring as code by automatically installing and configuring the Datadog Agent on nodes managed by Chef. Datadog's integration with Chef provides real-time visibility into what is happening with Chef resources, run-time performance, and execution failures. You can actively monitor the health of your Chef server in conjunction with the systems it’s managing, and be notified of any performance problems or anomalies.

Chef is a platform that automates configuration management for your infrastructure, supporting a continuous delivery workflow. It monitors the state of resources across your infrastructure to ensure each resource is in the desired state with every Chef run. Chef uses policies to manage workflows and operational requirements, applying configurations to machines based on their server type or where they fit in an organization's development process. The platform operates on three foundational elements: nodes, a workstation, and the Chef server. Nodes are managed by Chef, while the workstation is used to create configurations for your infrastructure. The Chef server acts as a hub for all your infrastructure's configurations and communicates between your workstation and each node. Chef enables you to implement infrastructure as code, efficiently manage all of the nodes across your infrastructure, and implement monitoring as code using its integration with Datadog. This integration provides real-time visibility into what is happening with your Chef resources, run-time performance, and execution failures. By deploying the Datadog Agent on your nodes using Chef roles, you can collect system-level metrics for your nodes by default and configure specific integrations such as Prometheus for collecting custom metrics.

This article discusses how to use Datadog to capture and monitor HAProxy logs and key metrics identified in Part 1 of the series. It provides step-by-step instructions on integrating Datadog and HAProxy, configuring the Agent, verifying the configuration settings, turning on the integration, viewing HAProxy metrics, capturing HAProxy logs, and creating alerts. The article also highlights how to use log management features in Datadog to collect and view HAProxy logs.

SQL Server is a relational database management system developed by Microsoft that uses Transact-SQL (T-SQL) as its query language. It offers various ways to optimize and configure resource use, such as storing tables in memory instead of on disk and controlling the distribution of CPU, memory, and storage through resource pools. To ensure optimal performance, it is essential to monitor key metrics related to SQL Server's functionality, including data persistence, query optimization, indexes, and resource pooling. This guide provides an overview of these metrics and their importance in maintaining a healthy and available SQL Server environment.

SQL Server offers various monitoring tools and features to optimize resource usage and gain insights into database health and performance. Tools like T-SQL queries, SQL Server Management Studio (SSMS), and Windows Performance Monitor enable users to gather, visualize, and analyze metrics in real time. T-SQL queries allow flexible data collection through dynamic management views and built-in functions, which can be integrated into automated monitoring solutions. SSMS provides a graphical environment to monitor system activity, visualize query execution plans, and generate reports, although with limited interactivity. Performance Monitor, a built-in Windows tool, helps correlate SQL Server performance counters with system-level metrics. These tools collectively provide comprehensive monitoring capabilities, aiding in database management and performance optimization. The series will further explore using Datadog for advanced monitoring and visualization in subsequent parts.

The article details the process of integrating Datadog with SQL Server to monitor system performance, offering insights into metrics, distributed request traces, and logs within a unified platform. It guides users through installing the Datadog Agent, specifically on Windows, and configuring it to track SQL Server instances, even those on remote Linux machines. The integration allows users to visualize SQL Server performance through customizable dashboards and provides query-level data through distributed tracing, enabling detailed analysis of application requests and performance. Additionally, the article highlights the capabilities of Datadog's log management features, which can correlate logs with metrics and traces for comprehensive monitoring and troubleshooting. By leveraging these tools, users can gain deeper insights into their SQL Server interactions and address any performance issues effectively.

SQL Server is a relational database management system (RDBMS) developed by Microsoft for Windows and Linux. It uses Transact-SQL (T-SQL), which can be compiled into batches of statements that SQL Server caches to improve query performance. The system allows configuration of resource use, including memory, CPU, and storage allocation through resource pools, virtual instances of SQL Server that access a slice of their parent's resources. Key metrics for monitoring include batch requests per second, last elapsed time, SQL compilations per second, SQL recompilations per second, buffer cache hit ratio, page life expectancy, checkpoint pages per second, used memory, CPU usage percentage, disk read and write I/Os per second, page splits per second, avg_fragmentation_in_percent, user connections, and system resource utilization. Monitoring these metrics can help diagnose issues with query performance, resource allocation, and database fragmentation, enabling optimization of SQL Server configurations for improved visibility into databases.

Datadog introduces a new feature called Container Map that provides real-time monitoring and debugging of containers in an environment. This tool complements other container monitoring capabilities like Autodiscovery, which detects newly launched containers, and Live Container view for drilling down to the process level. The Container Map offers a global overview of the container infrastructure, allowing users to group, filter, and explore all containers on the fly using metadata such as services, availability zones, roles, partitions, or any other dimension. It also integrates with various orchestration tools like Kubernetes, Amazon ECS, and Mesos for aggregating and filtering purposes. The tool enables users to zoom in on specific deployment groups, inspect containers' health and resource usage, and drill down to the process level for further investigation.

Datadog has announced a new feature called the "container map" which provides a global overview of a user's container environment, allowing for easy grouping, filtering and exploration of containers on the fly. The container map builds on Datadog's existing container monitoring capabilities, including Autodiscovery and Live Container view, to provide deeper visibility into modern infrastructure and applications. With the container map, users can easily identify issues such as resource usage and potential security risks, and troubleshoot problems by zooming in on specific containers or hosts. The feature also integrates with other orchestration tools like Kubernetes, Amazon ECS, and Mesos, allowing for aggregation and filtering of container data based on tags from those integrations.

Datadog has integrated with Prometheus, allowing users to send their metrics to Datadog using the OpenMetric exposition format. This integration enables users to analyze and visualize their Prometheus metrics alongside other data collected by Datadog, including application metrics and service checks. The integration is designed to be easy to use, requiring only a configuration file edit to start collecting metrics from a Prometheus backend. Once configured, users can build comprehensive graphs, dashboards, and alerts using the Prometheus data model, with labels automatically mapped to Datadog tags. Additionally, Datadog provides support for custom checks and Autodiscovery feature to monitor Kubernetes clusters and containerized services. The integration also notes that Prometheus metrics are subject to certain limits as custom metrics.