August 2015 Summaries

Amazon Kinesis is a managed service for handling large data streams such as clickstreams and social media feeds. It enables real-time processing and distribution of data to multiple consumers simultaneously. Datadog has recently added performance monitoring for Amazon Kinesis, allowing users to track key metrics like incoming and outgoing requests, latency, iterator age, and the evolution of shards per stream. Custom tags can also be applied to Kinesis streams for better metric organization. Alerts can be set up within Datadog to notify users of any abnormalities in these metrics.

Amazon Kinesis is a managed service for processing and managing large data streams in the AWS cloud. It allows users to ingest, process, and distribute data to multiple consumers simultaneously. Monitoring key performance metrics such as incoming requests, outgoing requests, latency, iterator age, and shards per stream can help ensure that Kinesis is performing at its best. Additionally, custom tags can be applied to Kinesis streams to split, aggregate, or filter metrics, and Datadog offers alerting features to notify users of any abnormalities with Kinesis metrics. The integration allows users to track performance metrics in real-time, providing valuable insights into the health and performance of their Kinesis applications.

Microsoft's Windows Management Instrumentation (WMI) is utilized to collect and manage performance metrics and counters for various popular Windows-based applications, including Exchange, SharePoint, SQL Server, and IIS. Datadog's WMI integration allows users to pull in these metrics from Windows applications for graphing, analysis, and alerting purposes. This integration also enables the creation of custom metrics within WMI for monitoring custom built .NET applications. Additionally, alerts and discrete events from Microsoft Event Viewer can be sent to Datadog's Event Stream for further analysis and correlation with other collected data. Datadog's WMI integration is particularly beneficial for engineering teams that use the full Microsoft stack and develop in .NET, as it allows them to track and report on high-level .NET app metrics. The platform also supports aggregating Windows app data precisely through tagging, which can be associated with cloud platforms, configuration management tools, and other sources. This feature is especially useful for applications hosted on cloud platforms where underlying servers may scale dynamically. Overall, Datadog's WMI integration provides valuable visibility into the performance of Windows-based applications and custom .NET applications, allowing users to monitor, analyze, and optimize their systems effectively.

Datadog’s WMI integration allows users to collect and manage performance metrics and counters for popular Windows-based applications, including Microsoft Exchange, SharePoint, SQL Server, and IIS. With this integration, users can also track metrics for .NET applications and send alerts from Microsoft Event Viewer to Datadog's Event Stream. Additionally, the integration enables users to create custom metrics and tags for their .NET applications, allowing for precise aggregation of data in cloud environments. This feature is particularly useful for engineering teams that use the full Microsoft stack and develop in .NET, providing visibility into high-level app metrics and system performance.

This article discusses the monitoring of Azure virtual machines (VMs) and provides an overview of key metrics to track their health and performance. It focuses on Infrastructure-as-a-Service (IaaS) deployments, where Azure's basic compute resource is the VM. The four main types of metrics discussed are CPU usage, disk I/O, memory utilization, and network traffic. For each metric type, specific examples of relevant metrics are provided along with their descriptions and metric types. Additionally, the article suggests which metrics to alert on for each type. The conclusion emphasizes that monitoring these general-purpose metrics will provide a high-level view of VMs' health and performance, while specialized metrics may be needed over time depending on specific applications.

This article discusses the integration of Azure virtual machines with Datadog to provide a comprehensive view of infrastructure metrics and application-level metrics. It explains how to set up this integration using two methods - enabling Datadog to collect metrics via the Azure API or installing the Datadog Agent using the Azure web portal. The benefits of each method are outlined, including the ability to customize dashboards and monitor additional server-level metrics from the host. Finally, it highlights how monitoring Azure with Datadog can provide critical visibility into VM performance and enable automated alerts on any metric across any group of VMs.

You can integrate Azure with Datadog to collect and view metrics from across your infrastructure, correlate VM metrics with application-level metrics, and slice and dice your metrics using any combination of properties and custom tags. You can use the Datadog Agent to collect more metrics—and at higher resolution—than are available in the Azure portal. This integration provides a comprehensive view of your infrastructure, allowing you to monitor individual VMs as well as entire applications. By connecting Datadog with Azure, you can easily create automated alerts on any metric across any group of VMs, with triggers tailored precisely to your infrastructure and usage patterns. The integration also enables the installation of the Datadog Agent on an Azure VM, providing system metrics for each VM, as well as application metrics that can be correlated with host-level metrics. This allows you to gain critical visibility into what's happening with your VMs and applications, enabling data-driven decisions to improve performance or save costs.

John Matson discusses different methods for capturing and monitoring Azure metrics, focusing on using the Azure web portal and accessing raw metric data via Azure storage. Users can graph and set alerts on performance metrics directly in the Azure web portal, with the option to enable the Diagnostics extension for more detailed data collection. The web portal allows users to select various metrics and set alert rules for specific thresholds. Additionally, the article explains how to access raw metric data stored in Azure storage tables using tools like Visual Studio's Cloud Explorer and AzCopy, emphasizing the capability to export data for external analysis. The post is part of a series, with a subsequent entry detailing integration with Datadog for enhanced monitoring of Azure virtual machines.

AWS ElastiCache is a fully managed in-memory caching service that allows users to choose between Redis and Memcached as its backend caching engine. It serves as a drop-in replacement for running these data stores yourself. Since a cache can have critical impact on an application's performance, it needs to be continuously monitored. Datadog has introduced a new ElastiCache integration that enables users to monitor ElastiCache metrics along with native metrics from the backing software: Redis or Memcache. This complete set of metrics provides a full view of your cache and includes key metrics such as number of commands processed, cache hits and misses, evictions, swap usage, current connections, number of nodes per cluster, replication lag, and CPU utilization. Additionally, native metrics for greater resolution are available directly from Redis or Memcached, providing more granular information about memory usage, memory fragmentation ratio, and latency. Custom tags can be used within Datadog to split, aggregate, or filter metrics, and all the events related to ElastiCache and your caching engine are recorded in Datadog's event stream for correlation with changes in performance throughout infrastructure.

Jean-Mathieu Saponaro from We published more recently a complete series of posts about the top ElastiCache and native cache performance metrics, how to collect them, and how Coursera monitors them. Application caches can greatly improve throughput and reduce latency of read-intensive workloads by boosting the performance of media transfers, games, or social networking applications. AWS ElastiCache is a fully managed in-memory caching service that allows you to choose between Redis and Memcached as its backend caching engine. The integration with Datadog provides a complete set of ElastiCache metrics along with native metrics from the backing software: Redis or Memcached. Default Elasticache integration dashboard displays key ElastiCache metrics, such as throughput measurement, cache hits and misses, evictions, swap usage, current connections, number of nodes per cluster, replication lag, CPU utilization, memory usage reports, memory fragmentation ratio, latency, custom tags, and correlating metrics with cluster-level events. The native metric collection provides real-time and high-resolution data for key performance indicators such as cache hits, misses, evictions, replication lag, current connections, memory usage, memory fragmentation ratio, and latency. Additionally, the integration supports correlation of changes in ElastiCache infrastructure with changes in performance throughout your infrastructure via Datadog's event stream.