May 2016 Summaries

The text discusses the integration of Datadog with Statuspage, a tool used by many external services to communicate their availability status to customers. This integration allows users to receive notifications when the services they are following experience downtime, and provides the capability to overlay these notifications with other metrics in their event stream for better context. Users can annotate and discuss incidents directly within the event stream, which facilitates faster investigation and resolution of issues. The setup for this integration is straightforward, requiring only a few minutes within the Datadog app, and allows users to tag and track the services they monitor efficiently. Existing Datadog customers can easily implement this integration to enhance their infrastructure monitoring, while new users are encouraged to start a trial to explore these capabilities.

This article discusses three methods to collect MongoDB metrics from your hosts: using utilities offered by MongoDB, using database commands, and using a dedicated monitoring tool for more advanced features. The two main utilities are mongostat and mongotop. Database commands include serverStatus, dbStats, collStats, getReplicationInfo, replSetGetStatus, sh.status, and getProfilingStatus. For production monitoring, MongoDB's management tools like Ops Manager and Cloud Manager can be used. Datadog also offers an integration for extended monitoring functionality.

This article discusses the monitoring of MongoDB performance metrics using the MMAPv1 storage engine. It provides an overview of NoSQL databases and explains how MongoDB works as a document-oriented database. The key areas to track and analyze metrics are outlined, including throughput metrics, database performance, resource utilization, and concurrent operations management. Metrics related to replication and oplog, journaling, background flush, cursors, storage size, memory usage, and host-level metrics are also discussed. The article concludes by emphasizing the importance of monitoring these metrics for maintaining good MongoDB performance and identifying areas where tuning could provide significant benefits.

You can monitor MongoDB performance with the MMAPv1 storage engine by tracking throughput metrics, database performance, and resource utilization. Key areas to focus on include read and write operations, replication and oplog metrics, journaling, concurrent operations management, and errors. Monitoring these metrics can help you identify potential issues before they become user-facing problems, allowing you to optimize MongoDB's performance and ensure high availability. By using a combination of tools such as MongoDB's utilities, commands, or dedicated monitoring tools, you can collect and analyze metrics to gain visibility into your database's health, performance, resource usage, and potential areas for tuning.

Monitoring MongoDB performance with the WiredTiger storage engine requires tracking various metrics to ensure high availability and efficient resource utilization. Key areas to focus on include throughput metrics, database performance, replication and Oplog, journaling, concurrent operations management, cache metrics, errors, and scaling MongoDB using sharding or replication. By monitoring these metrics, you can quickly spot slowdowns, hiccups, or pressing resource limitations and take corrective actions before they impact user-facing services.

The MongoDB database provides several methods to collect performance metrics from the host machine and from the database itself. The first method is using built-in utilities such as `mongostat` and `mongotop`, which provide real-time statistics on activity, memory usage, and storage information. These tools are useful for quick checks but do not offer advanced monitoring features. Another method is to use native MongoDB commands such as `serverStatus`, `dbStats`, `collStats`, `getReplicationInfo`, and `replSetGetStatus`, which provide a wide range of metrics on connections, operations, storage, and replication. For databases running in production, a more comprehensive monitoring system that ingests MongoDB metrics along with metrics from other technologies is often necessary, and tools such as MongoDB's own management tools and Datadog offer extended monitoring capabilities.

In the final part of a series on monitoring MongoDB, the article guides users through the process of integrating MongoDB with Datadog to effectively monitor and visualize database metrics. The post outlines a simple three-step process: installing the Datadog Agent, configuring it with a MongoDB-specific YAML file, and verifying the setup. It emphasizes the importance of creating a robust monitoring system for production databases, highlighting the ability to track and visualize key performance metrics, set up alerts, and integrate with communication tools for timely notifications. The article concludes by encouraging users to leverage Datadog's capabilities for enhanced visibility and proactive database management, offering a free trial for those without an account.

Datadog has integrated with Apache Hadoop, allowing users to immediately start monitoring four key technologies: HDFS, MapReduce, YARN, and Spark. This integration provides visibility into various metrics such as data node status, disk space usage, job performance, and more, enabling easier collaboration and problem-solving for distributed systems running on many machines. With the integration turned on, users can set alerts for critical issues and gain insights into their Hadoop ecosystem, making it a valuable tool for companies with large amounts of data to process. The integration is available for both existing Datadog users and new sign-ups, providing an easy way to monitor and optimize Hadoop performance.

Flowdock is an efficient tool that integrates with Datadog to streamline communication within technical teams during emergencies. By automatically receiving alerts, integration statuses, and more directly into your flows, teams can quickly begin problem-solving. The integration compiles related alerts from Datadog into their own Flowdock thread, keeping the main flow uncluttered for discussion. Manual posting within the Datadog app is also possible by tagging @flowdock-mention to highlight service anomalies. This integration helps teams efficiently pinpoint issues in a single thread while minimizing alerting noise. Setting up Flowdock integration with Datadog takes just a few minutes and can be done through the integrations tab within the Datadog app.

EC2 introduces multiple changes to software development, deployment, and maintenance processes due to its flexibility, ease of deployment, instant scalability, and vast ecosystem of third-party services. However, EC2 functions differently from traditional on-premise servers, leading to novel performance issues that require different tools to gain visibility into an application and its underlying cloud-based infrastructure. The five most common performance issues in EC2 include unpredictable EBS disk I/O, EC2 instance ECU mismatch and stolen CPU, running out of EC2 instance memory, ELB load balancing traffic latency, and AWS maintenance and service interruptions. To detect these issues, one can use CloudWatch metrics such as VolumeQueueLength for EBS volumes, CPU Utilization for EC2 instances, Request Count for ELB, Healthy Host Count for ELB, and Latency for ELB. Problem avoidance and resolution strategies include selecting the right storage and instance types, adding swap volumes to EC2 instances, programming applications in a "server-disposable" manner, creating new ELB instances, and regularly checking the AWS account console and status page. Datadog can help with AWS EC2 performance issues by providing fast and easy graphing and alerting of EC2 performance metrics, automatic registration and categorization of new hosts, customizable alerting for AWS EC2, and slicing and dicing of performance metrics for analysis.

John Matson's article, the second in a series about visualizing monitoring data, delves into summary graphs as a tool for providing a snapshot of infrastructure metrics over a specific time span. It explains aggregation across time, which compresses time series data into single values, and aggregation across space, which allows metrics to be grouped by various infrastructure attributes. The piece introduces different types of summary graph visualizations: single-value summaries, toplists, change graphs, host maps, and distributions, each with distinct purposes such as highlighting key metrics, spotting outliers, tracking changes, and visualizing infrastructure at a glance. By demonstrating when to use each type of visualization, the article aims to enhance the clarity and actionability of dashboard information, setting the stage for a subsequent discussion on avoiding common pitfalls in metric visualization.