August 2016 Summaries

AWS Redshift is a fully-managed data warehouse designed to handle petabyte-scale datasets with its low price, easy integration, and speed due to columnar data storage, zone mapping, and automatic data compression. Datadog's Redshift integration allows users to collect and visualize overall Redshift health and individual cluster health metrics for optimal performance. The out-of-the-box dashboard provides visualization of important metrics such as database connections, network throughput, read/write latency, and disk space usage. Monitoring these metrics helps identify potential issues like low disk space or reduced performance, which can be addressed by adjusting Workload Management (WLM) configuration, adding system resources, or changing node types. Custom alerts can also be set up to notify users of important changes in any of the cluster metrics.

AWS Redshift is a fully-managed data warehouse designed to handle petabyte-scale datasets, offering low price, easy integration with other systems, and speed thanks to columnar data storage, zone mapping, and automatic data compression. Datadog's Redshift integration allows for automatic collection and visualization of overall cluster health and individual metrics, enabling users to ensure optimal performance. The integration provides a range of key metric groups to monitor, including disk usage, network throughput, and IOPS, as well as tools for addressing reduced performance through configuration adjustments and node upgrades. By setting up custom alerts and correlating Redshift metrics with infrastructure metrics, users can stay ahead of potential issues and quickly identify the root cause of problems. With Datadog's integration, existing customers can easily enable metric collection, while new users can sign up for a free trial to start monitoring their clusters today.

Puppet is a widely used configuration management/orchestration tool that helps define, manage, and deploy changes across infrastructure. It uses a server/client architecture where the primary server compiles catalogs based on facts from agent nodes to enforce desired configurations. Datadog's integration with Puppet allows users to monitor performance metrics and events, enabling them to find out when Puppet runs occur, track run times, see resource changes, set alerts for failed updates, and correlate issues with other parts of the infrastructure. The Puppet-Datadog module also helps configure the Datadog Agent on primary server and agent nodes, allowing users to monitor more than 30 integrations.

Puppet is a widely used configuration management and orchestration tool that simplifies defining, managing, and deploying changes across an entire infrastructure. Developed in 2005 by Luke Kanies, Puppet is written in Ruby and available as both open source and commercial software, with over 30,000 companies using it to deploy and orchestrate their applications. Datadog's integration helps monitor Puppet performance metrics and events, allowing users to track run times, resource changes, failed updates, and other key metrics, and set alerts for potential issues. The integration also enables correlation of Puppet runs with other infrastructure metrics, providing a more comprehensive view of the overall system health. Additionally, Datadog customers can configure Puppet to automatically install and configure the Datadog Agent on nodes, allowing for seamless monitoring and tracking of Puppet performance.

At Datadog, a team of makers and hackers decided to solve the issue of bathroom contention in their office space by implementing a monitoring system. The goal was to create a non-invasive, reliable, and professional solution that would indicate whether a bathroom was occupied or not without being creepy. They used Raspberry Pi 2 Model Bs as the brains of the project, along with various sensors such as magnetic reed switches, pin switches, and photoresistors to detect door lock status and room availability. The team managed to set up a system that allowed them to check bathroom status via network-connected devices or even add it as a widget on their Datadog dashboards throughout the office.

The Datadog team, led by Chris Hoey and Daniel Benamy, aimed to create a system to monitor bathroom availability in their office. They wanted to avoid using video or sensors that could be perceived as intrusive, instead opting for magnetic reed switches and pin switches to detect lock positions. The team chose Raspberry Pi 2 Model Bs as the brain of the project due to their ability to run Linux and connect via WiFi, which simplified device management. To read sensor values, they used the GPIO pins as files, making it easy to access sensor status through a python script. They also kept things simple by using Netcat for bathroom availability checks and added bathroom status widgets on their Datadog dashboards. The project required minimal code, with most work going into figuring out sensor types, mounting devices neatly, and troubleshooting WiFi issues.

The text discusses the use of Consul, a distributed system for service discovery and configuration management, at Datadog over an 18-month period. Key points include using Consul to distribute configurations across clusters and discover service endpoints in microservices architecture. Recommendations are provided on server sizing, enabling fast auditable configuration changes, utilizing Access Control Lists (ACLs) for data protection, preventing self-DDoS attacks with watches, leveraging dnsmasq for DNS load distribution, and monitoring Consul's health metrics.

At Datadog, three new regression functions have been introduced to enhance dashboards: trend_line(), robust_trend(), and piecewise_constant(). These functions help visualize trends in data by highlighting the overall direction of a metric over time. Trend lines can be added alongside original metrics to clarify trends or remove visual clutter from charts with "group by" queries. Piecewise regression is useful when a series cannot be well represented by a single regression line, as it fits a step function that shows sudden changes in a metric's value. These functions are now available through the graph editor for all Datadog users.

Darron Froese, a Datadog engineer, shares his team's experience with using Consul for about 18 months in their production stack. Consul helps distribute configuration and discover service endpoints across the cluster. Key recommendations include: giving server nodes more CPU power to minimize leadership transitions, using git2consul for fast auditable configuration changes, utilizing Access Control Lists (ACLs) to secure data, monitoring Consul with metrics such as leader reconciliation count, DNS query count, and CPU/networking usage to ensure a healthy cluster. Additionally, using dnsmasq can help lighten the load on Consul by caching DNS queries and reducing the number of requests hitting Consul directly.

Datadog has introduced three regression functions to enhance its dashboards: `trend_line()`, `robust_trend()`, and `piecewise_constant()`. These functions help users analyze trends over time, rather than focusing on exact values. The `trend_line()` function uses ordinary least squares linear regression, while the `robust_trend()` function is more robust to outliers using Huber loss. Trend lines can be used to highlight trends, remove visual clutter from charts, and visualize changes in metrics before and after deployments. Additionally, the `piecewise_constant()` function can fit a step function to metric data, automatically determining segment duration based on metric characteristics. These functions are now available through the graph editor for all Datadog users.

Sumo Logic is a SaaS log management tool that aggregates and analyzes machine-generated events from various applications and infrastructure components. It can send parsed log data to Datadog, which displays the results as events in an event stream timeline for issue identification and collaboration. By correlating Sumo Logic events with performance metrics, users can gain a better understanding of their infrastructure's health. The integration also allows alerting on failed login attempts and customizing event message content. Additionally, Sumo Logic can receive events from Datadog by mentioning @sumologic-[connection-name] in any post or comment within the event stream. This enables users to leverage Sumo Logic's powerful parsing and searching capabilities for issue tracing and system performance analysis.

To integrate Sumo Logic with Datadog, set up a webhook connection in Sumo Logic and enter its URL in Datadog. Then select a scheduled search in Sumo Logic, which will be sent to Datadog as events, providing a log-like timeline for issue detection and collaboration. Correlating these events with performance metrics can help identify problematic applications or servers, while alerting on unauthorized login attempts can notify teams of potential security issues. Customizable event messages can provide actionable insight, and Sumo Logic can also receive events from Datadog to leverage its parsing and searching functionality. To get started, sign up for a Datadog trial account and follow the setup process in both tools.