October 2021 Summaries

Metrics without Limits™ is a feature by Datadog that allows users to cost-effectively collect, process, and archive custom metrics. It decouples metric ingestion from indexing, enabling users to dynamically specify which tags across metric names they want to index for analytics and troubleshooting. This helps maintain custom metrics volume within an observability budget without needing code-level changes or setting up separate proxy filters. Users can easily configure tags for individual metrics or groups of metrics, adjust their custom metric volumes by defining which metric aggregations they want to query, and retrieve archived data at any point for further analytics.

Datadog has introduced Metrics without Limits, a feature that decouples custom metric ingestion from indexing, allowing businesses to cost-effectively collect and process metrics. This feature enables users to dynamically specify which tags across metric names they want to index for analytics and troubleshooting without requiring agent or code-level changes. By doing so, users can reduce the volume of custom metrics emitted by their platform while maintaining full granularity for valuable tags. Users can easily configure tags for individual metrics or groups of metrics and fine-tune their indexing with aggregations. The feature also allows users to retrieve archived data, enabling them to re-index unindexed metrics at any point for further analytics. With Metrics without Limits, businesses can maintain visibility into their custom metrics while staying within their observability budget.

NS1 is an intelligent DNS and traffic management platform that optimizes network infrastructure performance and speeds application delivery to end users. It supports ECS DNS extension and Filter Chain technology, which minimize application latency and provide a high-quality user experience. Datadog's NS1 integration collects key metrics such as query rates and lease counts, visualizing them in an out-of-the-box dashboard for monitoring alongside other telemetry from the application infrastructure. The integration allows tracking of query rates across NS1 accounts, records, and zones, enabling adjustments to infrastructure or TTL configurations when necessary. It also provides visibility into Pulsar performance and availability, helping optimize routing and application delivery. Additionally, Datadog collects NS1 usage metrics for cost management and forecasting alerts to prevent exceeding monthly query limits.

NS1 is an intelligent DNS and traffic management platform that helps optimize network infrastructure performance and speed application delivery to end-users. It supports the edns0-client-subnet (ECS) DNS extension and Filter Chain technology, which minimize application latency over existing infrastructure. Datadog's NS1 integration collects key performance and usage metrics, such as query rates and lease counts, and visualizes them in an out-of-the-box dashboard, enabling a top-to-bottom view of services on a single platform. The integration also tracks query rates across zones, records, and accounts, and uses machine learning features to improve DNS visibility and detect potential issues. Additionally, Datadog provides visibility into Pulsar performance and availability, as well as NS1 usage metrics to help optimize costs.

The Datadog mobile app introduces widgets that allow users to create an on-call mobile dashboard directly on their phone's home screen for easy access to key data about the status and performance of applications. These mobile widgets provide quick visibility into the health of services, enabling users to monitor monitors, incidents, and SLOs without opening a laptop or even an app. The widgets can be combined into customizable on-call mobile dashboards that bring together monitoring data, collaboration tools, and communication channels for efficient incident response workflows. Additionally, focus mode helps users concentrate by temporarily silencing unrelated alerts during their on-call hours.

Datadog has introduced Network Device Monitoring, a device-first view that provides full visibility into every network component, allowing teams to identify issues before they impact their business. This new feature enables users to automatically discover and monitor thousands of network devices from various vendors, displaying key health and performance metrics in a comprehensive list. It also includes a timeseries graph of top bandwidth utilization by interface, making it easier to isolate problematic interfaces and track device performance. The feature provides granular details about individual devices and their interfaces, enabling teams to resolve issues before customers experience them. With Network Device Monitoring, Datadog breaks down silos between DevOps and Network teams, allowing them to work together to pinpoint the root cause of customer-facing issues.

USM provides comprehensive visibility into the health and performance of every service running on an organization's infrastructure, automatically detecting all services and monitoring their golden signals without requiring any code changes. This allows SRE teams to quickly respond to issues before they affect the organization, even as the fleet of applications expands. USM integrates with the Service Map to visualize upstream and downstream dependencies between services, making it easier to understand the cause of an incident. The Datadog Service Catalog centralizes information about all services, while USM provides RED metrics for creating alerts and SLOs for every service, enabling proactive monitoring of service health and performance. Additionally, USM offers faster troubleshooting with a unified platform that correlates logs, infrastructure metrics, and APM data to help teams identify the root causes of issues more quickly.

Datadog has introduced funnel analysis to its Product Analytics toolset, enabling users to visualize aggregated traffic across each step in key user journeys and monitor the rate of success for workflows. By leveraging this feature, users can identify sources of friction in their application's UI and determine where they need to optimize. Funnel analysis allows users to easily pivot to other features such as Session Replay to drill deeper into customer sessions and investigate the root causes of user friction. Users can add funnels to their dashboards alongside other key performance indicators for their application's health, making it easy to share product data across different teams within an organization. By analyzing funnel performance, users can understand the causes of user friction and make improvements to optimize their workflows.

At Dash 2021, Datadog introduced several new products and features aimed at enhancing visibility into various aspects of software performance and infrastructure management. Key announcements included advanced network device monitoring and Universal Service Monitoring, which offer insights into network components and backend services without code changes. Datadog expanded its Real User Monitoring (RUM) capabilities with features like iOS error tracking, Session Replay, and Watchdog Insights to better analyze user interactions and performance issues. The company also unveiled Datadog Observability Pipelines for managing data across infrastructures and Datadog Apps for integrating third-party tools directly into its platform. Additional updates included improvements in database monitoring, synthetic monitoring with private location and protocol tests, and new tools for log management such as Online Archives and Sensitive Data Scanner. Datadog also enhanced its developer experience with CI Visibility, mobile dashboard widgets, and a GitHub App, alongside cloud cost management solutions and serverless monitoring enhancements, showcasing its commitment to comprehensive, integrated observability solutions.

Datadog CI Visibility has partnered with CircleCI to provide enhanced visibility into CI/CD pipelines. The integration uses webhooks to capture information about the status and performance of workflows and jobs, enabling users to monitor pipeline performance, identify build failures, create pipeline-specific alerts, and reduce alert fatigue. This helps teams optimize their CI pipelines, troubleshoot issues more effectively, and improve overall efficiency in software development processes.

Datadog CI Visibility has partnered with CircleCI to provide a unified platform for monitoring and optimizing CI/CD pipelines. The integration uses webhooks to capture information about the status and performance of workflows and jobs, enabling users to monitor performance trends, identify inefficiencies, and troubleshoot build failures. With this visibility, users can optimize pipeline efficiency, improve reliability, reduce alert fatigue, and gain better insights into their CircleCI environment. The partnership provides a pipelines dashboard for tracking workflow performance, a built-in pipelines dashboard for comparing job performance, and pipeline-specific alerts to notify teams of issues. This integration aims to help teams release new features and bug fixes on time while reducing the number of environment issues that would otherwise cost more time and effort to address.

Steve Harrington is excited to release support for Azure App Service in the Datadog Serverless view, which provides a holistic view of an organization's App Service architecture. The view allows users to quickly get an overview of their App Service plans and resources, map relationships between resources in context with monitoring data, investigate performance issues, identify underutilized and overloaded plans, understand APM usage, and explore the health of their web apps and function apps. With this view, users can easily navigate their App Service plans at a glance, see key data from their plans, drill down to explore app-level metrics, traces, and logs, and pivot to other parts of the Datadog platform for further investigation without losing context. The view provides automatic APM tracing for Azure App Service apps and allows users to understand which plans are contributing to APM usage.

Datadog, a company that provides monitoring services for cloud-based applications, has contributed to the improvement of kube-state-metrics (KSM), an open source Kubernetes service. The contribution aimed at enhancing the scalability and extensibility of KSM by reducing the time spent collecting metrics and lowering memory and CPU footprint. This was achieved through a new design that allowed for better integration with Datadog's Cluster Check feature, leading to significant improvements in performance. The company also plans to introduce ways to register custom MetricFamilies for any Kubernetes resource and generate metrics for CRDs, allowing users to monitor custom resources more easily.

The Datadog Containers team contributed to the kube-state-metrics project, a popular open-source Kubernetes service that generates metrics about the state of objects in a Kubernetes cluster. The team was facing challenges scaling the tool to their needs, including high data volumes and performance issues. To address these challenges, they designed an extensible solution that utilized the Datadog Cluster Check feature. This solution allowed them to reduce network latency, memory footprint, and CPU usage, while also improving scalability and extensibility. The team's contribution introduced a new Kubernetes State Metrics check in the Datadog Agent, which runs a long-running thread that pulls data from the kube-state-metrics process. This resulted in significant performance improvements, including reduced execution time and memory footprint. The new solution enables users to monitor any custom resource and generate metrics for CRDs, providing a seamless experience for internal Datadog users and contributing back to the upstream code base.

AWS Lambda functions can be monitored by collecting and visualizing JSON request and response payloads using Datadog. This helps identify the root causes of function failures, such as misconfigured requests or code issues within the function itself. Datadog also provides the option to scrub sensitive data from collected payloads. Additionally, tracing serverless applications with Datadog allows for quicker troubleshooting by tagging spans that represent function invocations with relevant payload data. Payload ingestion is currently available for Python and Node.js runtimes and can be enabled by adding the environment variable DD_CAPTURE_LAMBDA_PAYLOAD to Lambda functions and setting it to true.

Datadog has introduced the ability to collect and visualize JSON-formatted request and response payloads of AWS Lambda functions, providing deeper insight into serverless applications and helping troubleshoot problems. This feature allows users to quickly identify misconfigurations in failing requests and view function responses to better understand issues that may occur. Datadog also provides an option to scrub sensitive data from collected payload and tag spans with relevant data from request and response payloads, accelerating troubleshooting by allowing users to search and filter real-time and historical traces without additional code changes or configuration.

Datadog introduces new features for its Template Variable workflow, making it easier and more efficient to create dynamic, shareable dashboards. The new template variable modal allows users to create and organize variables quickly, while the available values field enables users to specify which tag values appear in the dropdown for a template variable. These enhancements help users focus on relevant data and filter out noise, improving their overall dashboard experience.

Datadog has introduced new features for template variables in their dashboards, making it easier to create and manage dynamic, shareable dashboards. The new template variable modal allows users to select tags or attributes for variables, automatically filling in the variable name with the tag key. Users can also set default values and drag variables to reorder them, making the variable set easier to parse. Additionally, Datadog now offers an available values field, which enables users to focus on relevant values and filter out noise, such as unsupported browsers or deprecated services. These new features aim to optimize the template variable workflow, allowing users to easily visualize and monitor the data they need.

Datadog's RUM iOS SDK enables developers to collect comprehensive crash data from their iOS apps, allowing them to track, triage, and debug recurring issues. By using the SDK, developers can visualize crash data in dashboards, analyze symbolicated error reports, and identify key information about app resources throwing errors, most common error types, and iOS versions experiencing crashes. Additionally, Datadog's Error Tracking feature groups similar errors together into issues, providing debugging info from the app alongside user session info, helping developers understand the scope of crashes and focus their debugging efforts on the most severe and frequent issues. With this setup, developers can easily track, triage, and debug application crashes to manage their impact on users and reduce churn.