August 2022 Summaries

The Datadog Kubernetes Overview Page provides a comprehensive view of an organization's Kubernetes resources and helps users monitor and troubleshoot their Kubernetes environment. It offers guidance on five common issues in Kubernetes clusters, quick navigation to relevant dashboards, and the ability to view and enable Recommended Monitors. Additionally, it links to external resources for further learning about best practices for monitoring and managing Kubernetes with Datadog. This tool is useful for both new application developers and seasoned platform engineers, allowing them to effectively monitor their containerized applications.

Snowflake is a data platform that enables users to store, manage, analyze, and share high volumes of structured and semi-structured data. It offers a single, persistent storage system for all data types and has several mechanisms to protect and retain data. Datadog's integration with Snowflake provides full visibility into the architecture, allowing users to optimize storage usage, monitor warehouse performance and compute credit consumption, and detect misconfigurations and security threats. The multi-tiered storage system allows for flexibility in data migration and ingestion while ensuring data protection through Time Travel and Fail-safe mechanisms. By monitoring organizational metrics and using anomaly monitors, users can control costs and optimize performance. Additionally, Datadog's integration with over 500 other technologies provides a complete picture of all data-related activity in the system.

The Kubernetes Overview Page is a single pane of glass that equips users with the knowledge to monitor and troubleshoot their Kubernetes environment. It offers useful guidance at every stage, helping newer users learn about the infrastructure hosting their team's applications. The page displays a bird's-eye view of the top-level resources from all clusters, including ReplicaSets, deployments, and services, allowing users to filter and explore in further detail. The page also provides tips for troubleshooting common issues such as pods in symptomatic phases, container restarts, and unschedulable nodes. Additionally, it offers a wealth of information through out-of-the-box dashboards that can be used to quickly monitor specific sectors of the Kubernetes environment, enabling users to gain valuable insights into their containerized applications. The page also enables users to enable Recommended Monitors for critical alerts and provides links to resources such as best practices guides, documentation, courses, and blog posts to help learn more about monitoring and managing Kubernetes with Datadog.

Snowflake is a cloud-based data platform that enables users to store, manage, analyze, and share high volumes of structured and semi-structured data. It breaks down the barriers between siloed data sources, serving as a single source of truth for various workloads. Monitoring Snowflake is crucial for optimizing cost, performance, and data quality. Datadog's updated integration provides full visibility into Snowflake's architecture, enabling users to monitor query and Snowpark performance, manage costs, stay ahead of security vulnerabilities, monitor data quality and usage, bring Snowflake data into Datadog, and get full visibility into the entire Snowflake architecture. The integration offers features such as monitoring warehouse size and memory capacity, detecting data quality issues, and streaming custom metrics and tags from Snowflake. With Datadog's capabilities, users can gain a complete picture of their Snowflake usage, identify potential security threats, optimize costs, and make informed decisions about their data management strategy.

Microsoft Azure has introduced new virtual machines powered by Ampere Altra Arm-based processors to provide efficient and cost-effective processing power. These VMs are designed to improve performance and reduce costs for scale-out and cloud-native workloads. Datadog, a monitoring service, can be used to gain full visibility into both Arm- and x86-powered Azure infrastructure, enabling users to evaluate performance and ensure successful migration of workloads. The integration provides out-of-the-box visualization of resource usage across the entire environment, as well as support for the Datadog Agent on both architectures. Additionally, Datadog APM allows for monitoring of end-to-end traces from Arm-hosted services to ensure proper handling of demand and expected performance boosts.

Microsoft Azure has introduced its new series of Azure Virtual Machines powered by Ampere Altra Arm-based processors, offering excellent price-performance for scale-out and cloud-native workloads. To get full visibility into these architectures, customers can use Datadog to collect metrics from their entire fleet of virtual machines, including Arm-based VMs, and visualize them in an out-of-the-box dashboard. The Datadog Agent supports both x86 and Arm-based architectures and provides additional insight into VM performance, down to the process level. With Datadog APM, customers can collect end-to-end traces from their services running on Arm-hosted servers to monitor request latency and error rates, ensuring optimal performance and cost savings. Additionally, Datadog's Azure integration provides unified out-of-the-box visibility into every layer of the Azure environment, enabling customers to easily evaluate performance and ensure workload continuity.

Akamai is one of the largest CDN solution providers globally, offering its Intelligent Edge Platform to help companies deliver content securely and efficiently. Datadog has integrated with various Akamai visibility tools, including Datastream and mPulse, providing insights into the health and performance of a company's CDN. The latest integration is with Akamai Datastream 2, which captures raw performance and security logs from edge servers and streams them to a destination of choice. This integration allows users to visualize key performance indicators such as request traffic, HTTP response codes, latency, cache hits and misses, and more in the Log Explorer and customizable dashboards. By centralizing logs and metrics together, users can quickly spot performance issues and take necessary actions to improve CDN performance.

Akamai is one of the world's largest CDN solution providers, helping companies accelerate the secure delivery of content globally through its Intelligent Edge Platform. Datadog integrates with Akamai's visibility tools, including Datastream and mPulse, enabling users to gain insight into their CDN's health and performance. The integration now also supports Akamai Datastream 2, a low-latency API service for log delivery that captures raw performance and security logs from edge servers and streams them to a destination of choice. With this integration, users can visualize Datastream 2 logs in the Log Explorer, create customizable dashboards, and use Datadog's log processing and analytics to generate new metrics. The integration also allows users to extract key metrics from their Datastream 2 logs before they leave their environment using Observability Pipelines, providing control over log volume and costs. This integration enables users to quickly spot performance issues, such as a low cache hit ratio, and get started today with a 14-day free trial.

The text discusses Edgecast, a global network platform that provides content delivery network (CDN) and other solutions for edge computing, application security, and over-the-top video streaming. It also introduces Datadog's Edgecast integration, which allows users to visualize key performance metrics with an out-of-the-box dashboard and get alerted on important issues such as low cache hit rate ratio or anomalous increase in traffic and errors. The integration includes a dashboard that displays key performance and traffic metrics, enabling users to monitor the impact of cache configuration changes in pre-production and deploy with confidence. Additionally, it allows users to visualize their Edgecast metrics in context with other services they integrate with. Users can also configure alerts for key thresholds for metrics like cache hit ratio and average bandwidth, as well as anomaly monitors to detect abnormal behavior that deviates from historical baselines.

The Edgecast platform provides a content delivery network (CDN) and edge computing solutions for improving web performance, application security, and over-the-top video streaming. With its integration with Datadog, users can visualize key metrics such as cache hit ratio and bandwidth, enabling them to analyze aggregate data and identify specific issues. The Edgecast dashboard allows users to monitor long-term shifts in their network, including changes in cache hit ratio, and provides insights into how policy changes affect caching performance. Additionally, users can alert on key thresholds for metrics like cache hit ratio and average bandwidth, as well as configure anomaly monitors to detect abnormal behavior that deviates from historical baselines. This integration enables users to gain visibility into their CDN operations, monitor the impact of cache configuration changes, and deploy with confidence.

DevOps teams often face challenges in troubleshooting incidents due to the sheer volume of log data, which can lead to delays in issue resolution and a degraded user experience. Zebrium addresses this by using unsupervised machine learning to identify root causes of incidents in logs, boasting a 95% accuracy rate according to a Cisco study. Zebrium has recently integrated with Datadog, allowing users to purchase a subscription through the Datadog Marketplace and enabling data streaming and alerting between the two platforms. This integration enhances incident response workflows by allowing the Zebrium app to enrich Datadog dashboards with root cause information, expediting troubleshooting without the need for manual log searches. An example involving an e-commerce site running on Amazon EKS illustrates how the Zebrium widget helps pinpoint the root cause of issues like network traffic drops, thereby reducing mean time to resolution (MTTR) and improving operational efficiency. The Zebrium app and integration are now available on the Datadog Integrations page, offering a streamlined process to identify and address system failures more effectively.

Google Cloud's Dataflow is a fully managed stream and batch processing service that simplifies data pipeline development using Apache Beam. Datadog has introduced an integration for Dataflow, allowing users to monitor all aspects of their streaming applications in one platform. The out-of-the-box dashboard displays key Dataflow metrics and other complementary data, providing comprehensive insights into Dataflow pipelines. With this integration, users can monitor the state of their pipelines, get notified about critical changes, understand upstream and downstream dependencies, and investigate root causes of issues. Datadog also integrates with over 650 other services and technologies, allowing teams to monitor their entire stack using one unified platform.

Google Cloud's Dataflow is a fully managed stream and batch processing service that offers fast and simplified development for data-processing pipelines written using Apache Beam. It has a serverless approach, removing the need to provision or manage servers, allowing developers to focus on programming instead of managing infrastructure. Dataflow also features integration with other Google Cloud services such as Vertex AI and BigQuery, enabling users to process data in their pipelines and replicate pipeline data into BigQuery. Datadog's new Dataflow integration provides full visibility into the state of Dataflow pipelines, allowing users to monitor failures, long-running jobs, and data freshness analytics in a single pane of glass. The integration enables teams to create custom alerts to notify them when critical changes occur in their pipelines, understand upstream and downstream dependencies, and get notified about system lag and other issues that may arise in the pipelines. With Datadog's integration, users can leverage out-of-the-box dashboards and monitors to visualize key Dataflow metrics and other complementary data, and use detailed troubleshooting tools such as Execution details, worker logs, and job metrics to investigate and resolve issues. The integration also provides complete visibility into upstream and downstream dependencies, allowing teams to quickly identify the root cause of issues in their pipelines.

The Datadog Service Catalog helps centralize knowledge about an organization's services, improving collaboration, service governance, and incident response. It automatically detects APM-instrumented services and writes their metadata to a service definition before adding them to the catalog. Users can also create their own service definitions in JSON or YAML format to manually register new services or update existing ones with contact information, on-call schedules, and other key metadata. The Service Definition JSON Schema enables IDEs to autocomplete entries as users work with their service definition files, automatically validate data, and POST service definitions to the Datadog Service Definition API. Additionally, the schema's extensions field allows users to store custom metadata in their service definitions for programmatic access.

The Datadog Service Catalog is a centralized knowledge base for an organization's services, enabling collaboration, service governance, and incident response. The Service Definition JSON Schema helps create and update Service Catalog records efficiently by leveraging IDEs' capabilities such as autocomplete data in service definitions, automatic validation of service definitions, and posting service definitions to the Datadog Service Definition API. The schema also enables storing custom metadata in the `extensions` field, allowing teams to manage their Service Catalog records using existing tools and practices.

Datadog Synthetic Monitoring and Real User Monitoring (RUM) provide critical insights into the effectiveness of test suites by leveraging real user data, allowing teams to identify the most crucial workflows to test, untested actions, and optimize their test suite. The new Test Coverage page offers a comprehensive view of test coverage, enabling users to gauge the overall effectiveness of their testing strategy and pinpoint areas for improvement. By analyzing user action data, Datadog's Test Coverage page helps teams understand which tests are covering key workflows, identify gaps in their testing, and refine existing tests to ensure their application provides the best possible user experience.

The release of the Datadog Cluster Agent allows users to autoscale their applications running in Kubernetes in response to real-time fluctuations in any metric collected by Datadog. With the new CRD, users can customize their metric queries with functions and arithmetic, giving them more control over autoscaling behavior within their cluster. The Horizontal Pod Autoscaling (HPA) feature in Kubernetes allows users to autoscale their applications off of basic metrics like CPU or user-defined custom metrics collected from within the cluster. With external metrics support introduced in Kubernetes v1.10, users can now autoscale off of any metric from outside the cluster, including those monitored with Datadog.

Rails is a Ruby framework that employs the Model-View-Controller architecture for web application development, integrating databases like MySQL and PostgreSQL and web servers such as Apache and NGINX. This guide illustrates how to use Datadog for monitoring a Rails application running on Passenger, Apache, and MySQL by collecting metrics, request traces, and logs. The process involves installing the Datadog Agent, configuring log collection for different components, and setting up Rails application performance monitoring using the ddtrace gem. Datadog's APM provides insights into service-level performance by tracing requests across services, automatically generating dashboards for key metrics. Additionally, Datadog allows log collection from various sources, supports JSON-formatted logs through the Lograge gem, and enables custom dashboards and alerts for comprehensive monitoring. The guide also details enabling Datadog’s Apache and MySQL integrations for metrics and logs collection, facilitating efficient infrastructure monitoring and troubleshooting through Datadog's customizable tools.

Organizations across diverse industries aim to deploy stable applications that meet customer needs, often relying on the Datadog platform for comprehensive visibility into application health and performance. The Datadog Technical Solutions team utilizes tools like Real User Monitoring (RUM), Session Replay, and Error Tracking to efficiently manage and resolve customer support tickets. These tools help bridge information gaps by providing detailed insights into customer sessions, enabling the team to replay user journeys, identify the root causes of issues, and implement solutions swiftly. This approach enhances the team's ability to address support tickets promptly while maintaining customer privacy. By integrating these tools into their workflows, Datadog ensures rapid resolution of issues, allowing users to continue serving their customers effectively.

Seekret, a powerful observability platform that uses eBPF technology to autodiscover and visualize API assets, interconnections, and dependencies, is joining Datadog. This integration will provide customers with deeper API observability, governance, and automation across the entire API lifecycle. Seekret's features include tracking API behavior, validating best practices, generating tests and documentation, verifying compliance to company policy, and analyzing the impact of API changes within CI/CD pipelines. The collaboration between Datadog and Seekret aims to enhance existing APM and security products by leveraging Seekret's eBPF expertise.

In modern Kubernetes environments, the complexity is increasing as the average number of pods per organization has doubled over two years. Datadog's Kubernetes Anomalies feature helps platform and application engineers by automatically surfacing anomalies in their Kubernetes clusters. This tool scans the entire Kubernetes infrastructure to detect and surface meaningful anomalies, such as high percentages of unavailable replicas, unhealthy nodes, pending pods, OOM-terminated containers, and restarted containers. By using this feature, users can quickly investigate anomalies across their Kubernetes environment, improving incident investigations, reducing mean time to resolution (MTTR), and enhancing the end-user experience.

Eventarc is a Google Cloud offering that simplifies the creation of automated, event-driven workflows by ingesting and routing events between GCP products such as Cloud Run, Cloud Functions, and Pub/Sub. The recent launch of Datadog source in Eventarc allows customers to configure any Datadog monitor to trigger Eventarc-driven workflows, enabling auto-remediation of issues, gathering contextual data for incident response, and analyzing historical trends in triggered alerts. This integration helps teams improve application resilience by reducing development overhead and automating complex tasks.

The Datadog Service Catalog is an automated tool that consolidates knowledge of an organization's services, streamlining communication between various teams and improving service governance. It helps bridge the gap in knowledge about services, their dependencies, and ownership among team members. By providing a central repository of information about services, it assists in incident management, day-to-day operation and maintenance of services, and deployment updates. The Service Catalog also provides valuable insights to stakeholders across the organization, helping them manage communication during active incidents or plan reliability exercises. It is generally available and free for all Datadog customers.

The Datadog Service Catalog is a tool that helps organizations centralize knowledge about their services and improve communication between teams. By providing a shared repository of information, such as documentation, deployment history, and code libraries, the Service Catalog bridges the knowledge gap between new engineers and experienced teammates, allowing them to gain an understanding of their services and start contributing right away. The catalog also provides key data that helps teams manage incidents more effectively, deploy code updates with confidence, and provide service observability to the entire organization. This centralized repository of information is automatically populated for APM customers and can be manually registered by non-APM users, making it a valuable tool for simplifying service governance and improving application reliability.

Eventarc is a Google Cloud offering that simplifies the creation of event-driven workflows by automatically handling event ingestion, delivery, authorization, and error handling. The new Datadog source in Eventarc allows users to configure any Datadog monitor to trigger Eventarc-driven workflows, which can be used for auto-remediation, incident response, and analytics. With this integration, teams can automate incident response by kicking off complex workflows that perform tasks such as notifying responders and executing custom Cloud Functions. The integration also enables the collection of contextual data, such as metrics and logs, to help on-call engineers minimize context switching and reduce their mean time to resolve (MTTR). Additionally, it allows for alerting analytics, enabling users to surface historical trends and gain insights into the overall health and performance of their services. The Eventarc and Datadog integration is now available in a public preview for GCP customers, with detailed tutorials and a free trial available for new users.

The Datadog Software Catalog is a tool designed to centralize and streamline knowledge about an organization's software services, aiding in the transition from monolithic to microservices architectures. It consolidates information about service dependencies, ownership, and performance metrics, and is automatically populated with data from Datadog's APM customers, telemetry data, and other asset inventories like ServiceNow. This centralized repository enhances incident management, improves communication among engineers, and supports reliable code deployments by providing a clear view of service dependencies and performance. The catalog also facilitates collaboration by offering key information such as on-call schedules and service documentation, which allows for more efficient troubleshooting and operational management. It enables stakeholders across the organization to access critical data independently, aiding in communication during incidents and facilitating exercises like game days. The Software Catalog is part of Datadog's Internal Developer Portal product, with many features available for free to customers, and offers advanced capabilities for a more comprehensive understanding of service observability and operational practices.

Bowen Chen explains how GitHub Actions provides tooling to automate and manage custom CI/CD workflows from your repositories, allowing for high velocity build, test, and delivery of application code. Datadog CI Visibility offers end-to-end visibility into GitHub Actions pipelines, helping maintain their health and performance through metrics, distributed traces, and job logs. Integrating GitHub Actions with CI Visibility allows users to investigate failing pipelines and performance bottlenecks, track down suspect commits, view correlated log data, analyze historical trends in build duration, and remediate issues to focus on developing new features. The integration provides a more granular view into pipeline executions, enabling teams to quickly identify and fix problems, ultimately improving the overall delivery process.