June 2024 Summaries

The Teleport Access Platform is a foundation of cryptographic identity and zero trust that delivers on-demand, least-privileged access to infrastructure. This platform eliminates attack surfaces of shared secrets and standing privileges, improving efficiency, resilience, compliance, and security for engineering teams and applications. Datadog's integration with the Teleport platform enables monitoring of services and audit user session activity, providing a suite of metrics and an out-of-the-box dashboard for tracking critical service health and performance. The integration also closely audits user sessions via Teleport audit logs to detect unusual behavior, generates security signals with relevant audit logs for investigation, and provides comprehensive monitoring coverage for Teleport services. With this integration, users can ensure their Teleport services are working as expected, monitor access across entire infrastructure, and detect suspicious activity.

Golden Path project scaffolding aims to simplify the process of creating new projects by providing self-service tools and processes that help engineers start new projects in a more standardized, less mistake-prone way. Datadog Workflows and App Builder can be used in conjunction with cookiecutter to create Golden Paths for organizations, which include key template assets such as infrastructure and observability manifests, testing frameworks, documentation templates, and code formatting guidelines. By using these tools, developers can create new projects more efficiently, collapse knowledge silos, streamline onboarding, and make it easier for teams to get their services working together. The Golden Paths app can be used to generate new projects from a self-serve UI, making it easier for engineers to access templates and spin up new projects with fewer hiccups. Additionally, the app can be embedded in a dashboard to improve discoverability and convenience.

Datadog Database Monitoring (DBM) provides visibility into the performance of PostgreSQL instances and applications by enabling developers, SREs, and database analysts to get a shared view of insights beyond core telemetry. DBM allows users to explore data models and schemas, verify database configuration settings for fine-tuning, track database and query health, monitor database instance performance, analyze database load patterns, diagnose fleet-wide issues more quickly, and improve database and query performance with Query Insights. By providing all teams with this information, Datadog makes it easier to identify opportunities for optimization and respond to incidents more effectively.

The new indexing strategy uses an inverted index to associate every tag in a timeseries with the identifiers of timeseries that contain the tag. This allows for efficient query execution, as queries can be executed by making a single key-value lookup for each queried tag and retrieving a set of relevant timeseries identifiers. The previous strategy required scanning the data for the entire metric when only a small subset was requested, which could lead to full table scans and poor user experiences. The new strategy also introduces write and space amplification, as every unique timeseries identifier has to be stored multiple times, but this is not expected to become a problem due to disk space utilization being minimal. Intranode sharding allows for parallel execution of queries by fetching data from each RocksDB shard in parallel and then merging the results. The service was rewritten from Go to Rust, which resulted in significant performance improvements, with CPU-demanding operations being up to 6x faster. This change allowed Datadog to query higher cardinality metrics on the same hardware, reducing query timeouts by 99% and making the timeseries index nearly 50% cheaper to run.

Datadog's Sensitive Data Scanner (SDS) helps detect and redact sensitive information in telemetry data to maintain regulatory compliance and improve data security. The latest update allows users to apply SDS scanning rules directly using the Datadog Agent, enabling on-premise redaction of sensitive information before it is sent to downstream services. This feature supports nearly 90 out-of-the-box scanning rules and can be configured for specific host groups. Users can choose between the Agent and Observability Pipelines based on their business use case, with the Agent being suitable for most organizations and those sending logs exclusively to Datadog.

OpenTelemetry (OTel) is an open source observability solution that allows teams to monitor their applications and services in a standardized format. Datadog has announced support for OTel, enabling users to access the full capabilities of the OTel Collector while using Datadog's industry-leading observability solutions. The latest version of the Datadog Agent now embeds a fully configurable OTel Collector, allowing users to manage their fleet of embedded OTel Collectors remotely and onboard faster without manual configurations. This integration provides interoperability across open source and vendor-based observability stacks, enabling seamless management and access to comprehensive Datadog solutions alongside the open source capabilities of OpenTelemetry.

Datadog Cloud SIEM` offers a solution to the common problem of deploying new threat detections effectively by using `Detection-as-Code`, which enables security teams to test their detection rules in various ways, including `backtesting`, `unit testing`, and `simulation`. The `Historical Jobs` feature allows users to run their detections against historical logs stored in `Datadog`, providing essential insights into potential threats or anomalies identified within the associated logs. By using `Historical Jobs`, security teams can conduct thorough investigations of past events, uncover activity patterns, and understand the context of previous security incidents. This approach helps build confidence that new rules will generate valuable signals at the right time and in the right manner.

Datadog has announced new capabilities for Cloud Cost Management, making it easier to instrument and monitor serverless workloads across cloud providers. The company also introduced new network monitoring features, including Network Path, which allows users to visualize the individual hops taken by traffic between their network's sources and destinations. Additionally, Datadog has improved its platform features, such as Disaster Recovery and Fleet Automation, to help customers achieve their observability goals. The company has also expanded its support for cloud providers like AWS, Azure, and Google Cloud, enabling users to monitor and manage their costs across multiple platforms. Furthermore, Datadog has introduced new tools for log management, including Flex Logs, which allows users to store and analyze high-volume logs efficiently. The platform now also provides features such as Watchdog Explains, which helps users identify the root cause of anomalies on graphs, and Nested Queries, which enables complex analysis of metrics data. Additionally, Datadog has enhanced its disaster recovery capabilities, allowing customers to meet their observability availability and business continuity goals. With these updates, Datadog aims to provide a unified platform for monitoring infrastructure and troubleshooting problems across the entire stack.

Datadog has introduced remote bulk instrumentation of Lambda functions, now in Preview, to help address the challenges of ensuring end-to-end visibility into transaction-level data for production serverless applications. This feature enables organizations to add features like enhanced metrics and distributed tracing to any number of Lambda functions without requiring changes to their CI/CD pipeline or infrastructure-as-code configuration. With remote bulk instrumentation, teams can reduce the time spent modifying deployment pipelines or infrastructure configurations and get visibility into execution-level Lambda traces during critical incidents, even when serverless apps may not have initially been instrumented. The feature allows for granular control over instrumentation, enabling organizations to select which functions to enable bulk instrumentation for and exclude others. This streamlines serverless application management by automatically ensuring AWS Lambda functions are instrumented without needing changes to IaC or code pipelines.

Datadog's Product Analytics offering is designed to help users explore product usage and make meaningful product decisions. It provides a centralized location for visualizing user interactions, identifying friction points, and investigating potential UX changes. With features such as Heatmaps, Session Replay, User Segments, and User Retention, users can gain insights into their app's performance and identify areas for improvement. By leveraging concrete usage data and granular user analytics, users can answer critical questions about their product design and make data-driven decisions to drive growth and engagement.

AWS IAM Access Analyzer is a service that identifies overprivileged resources in your AWS environment and validates your IAM policies against established best practices. Datadog Cloud Infrastructure Entitlement Management (CIEM) now integrates with AWS IAM Access Analyzer to help detect permission gaps in your cloud infrastructure and determine next steps for remediation. Datadog CIEM helps identify and address entitlement risks across your cloud environment by continually scanning your cloud infrastructure to surface issues such as lingering administrative privileges, privilege escalations, permission gaps, large blast radii, and cross-account access. The integration allows you to see unused access findings directly in Datadog, which can create risk in your environment by granting excessive access to sensitive cloud resources. When a finding is detected, Datadog CIEM suggests a comprehensive downsized policy that incorporates all of AWS IAM Access Analyzer's unused access detections, so you can mitigate permission gaps across your entire environment by adopting the suggested policy.

Datadog provides unified visibility into application health and performance across the full development lifecycle for both front- and backend developers. The platform offers features such as Single Step Instrumentation, which enables one engineer to instrument all services in minutes, and Adaptive Ingestion Sampling, which automatically adjusts sampling rates to hit budgeted usage. Additionally, Datadog offers data observability features like Data Jobs Monitoring, which helps detect problematic Spark and Databricks jobs, and Data Streams Monitoring, which provides a unified end-to-end view of ingestion, processing, and storage. The platform also offers digital experience monitoring tools such as RUM Custom Vitals, which enables tracking frontend component performance based on real user activity, and Datadog Mobile RUM, which provides comprehensive crash reporting across iOS, Android, and React Native apps.

Datadog Cloud SIEM has integrated user and entity behavior analytics (UEBA) to detect insider threats, compromised accounts, and unusual behavior in dynamic cloud environments. The platform correlates alerts with key identity attributes and applies heuristic risk scores to minimize false positives and prioritize threats. Datadog Cloud SIEM Risk Insights for AWS and GCP entities now offers deeper behavioral and environmental context for investigations, integrating data from Datadog Cloud Security Management to assess the risk level of entities effectively. This centralized approach helps security teams focus on high-risk insights, improve their ability to respond promptly and effectively to real threats, and streamline investigations by consolidating correlated signals and providing actionable views of potential risks.

Reilly Wood from Datadog has announced the Preview of CoTerm, a solution designed to provide error-proof, team-powered command-line workflows. This tool offers real-time checks on sensitive commands in terminals, enabling teams to avoid missteps that can have major ramifications. CoTerm can quickly check for common errors in commands and prevent mistakes like running a risky operation without proper context. It also allows for lightweight approval workflows for sensitive operations, requiring a second pair of eyes before execution. Additionally, CoTerm records terminal output, which is browsable and replayable in Datadog, allowing teams to search through recordings to answer questions or share knowledge with coworkers. The tool works by shimming commands using the PATH environment variable, creating lightweight scripts and adjusting shell configurations as needed.

Nicholas Thomson, Jonathan Morin, Ryan Warrier, and Jane Wang discuss the challenges of monitoring data pipelines and the importance of gaining complete visibility into data's health at every stage of the process. They argue that traditional methods of monitoring only check on data quality as a proxy for pipeline efficiency, leaving room for improvement. Datadog's suite of solutions, including Data Streams Monitoring (DSM) and Data Jobs Monitoring (DJM), enables teams to monitor the entire data lifecycle from end to end, providing insights into both pipeline performance and data itself. With DSM, users can track streaming data pipelines, application dependencies, and storage components in a unified map, gaining visibility across services, queues, and jobs. DJM provides alerting, troubleshooting, and optimization capabilities for Apache Spark and Databricks jobs, allowing teams to optimize data job provisioning, configuration, and deployment. Datadog also offers new capabilities to monitor the data itself, including detecting and alerting on freshness and volume issues, analyzing table usage based on query history, and understanding upstream and downstream dependencies with table-level lineage. By integrating these solutions, teams can gain end-to-end visibility into their data pipelines, troubleshoot issues more effectively, and save hours of cross-team troubleshooting.

Datadog's Live Debugging streamlines the process of fixing production bugs by providing crucial context that helps developers quickly identify and reproduce the root cause. It visualizes the affected service, its dependencies, and the flow of data between them to pinpoint which interactions contributed to the error. The tool also enables AI-generated summaries and root-cause hypotheses, reproducing bugs locally with one-click integration test generation. By providing runtime context, including values of local variables at the time an exception was thrown, Datadog's Exception Replay enhances Error Tracking, making it easier to troubleshoot production bugs without leaving the usual workflow. The Debugger uses APM data to visualize the flow of requests between services and surfaces any errors that may have contributed to the problem. With Live Debugging, developers can efficiently investigate and fix production errors with AI insights and runtime context, regaining momentum on shipping next features.

The latest evolution of Bits AI is an autonomous agent capable of performing complex operational tasks without constant human prompting. It can investigate alerts, identify potential root causes, and drive towards resolution, while also coordinating incidents and providing key telemetry data. The new version uses large language models to reason, make decisions, and orchestrate processes, aiming to help developers operate more efficiently across the end-to-end DevSecOps lifecycle. Bits AI is now available in Preview and can be integrated with Datadog Incident Management for a 14-day free trial.

Datadog LLM Observability is a powerful tool designed to help AI engineers and software developers develop accurate, cost-efficient, secure, and highly performant Large Language Model (LLM) applications at scale. It enables end-to-end tracing of LLM application workflows, allowing users to monitor, secure, and improve their applications by analyzing traces to troubleshoot issues, monitoring operational performance, evaluating functional quality, tracking security exposures, and integrating with other Datadog tools for granular visibility into LLM behavior. By leveraging these features, users can identify root causes of issues, optimize chain components, detect prompt injections and security exploits, and form actionable insights about their application's health, performance, and security from a consolidated view.

Datadog Security offers advanced threat detection, vulnerability management, and application security features to help organizations secure their infrastructure and applications across various environments. The platform provides real-time threat detection for AWS Fargate and Windows environments, as well as continuous vulnerability scanning for Windows hosts. Additionally, Datadog Cloud Security Management (CSM) offers risk-based insights, historical job testing, and identity risk remediation, while Application Threat Management secures APIs with advanced security features such as exploit prevention and single-step instrumentation. These capabilities enable organizations to accelerate investigations, identify and remediate code-level vulnerabilities, and strengthen their overall security posture.

Datadog has introduced its On-Call feature, which aims to enhance the on-call experience for engineers by providing a single platform to observe the tech stack, detect issues quickly, and page the right people at the right time. The feature enriches the on-call experience with observability context, mobilizing responders with data-driven pages, service and team organization details, dynamic scheduling and notifications, and deep analytics for fast, purposeful coordination. Datadog On-Call consolidates monitoring and paging into a single platform, breaks down knowledge silos with clear team and service ownership, ensures timely responses with intuitive scheduling and escalation policies, and gains actionable insights from pages with detailed analytics. The feature is designed to simplify the on-call process, reduce stress, and improve overall incident response times.

Datadog is providing end-to-end network monitoring that enables teams to view the path taken by application traffic through different network segments, pivot to physical network devices for troubleshooting, and get in-depth monitoring of Cisco SD-WAN deployments. This visibility allows for faster troubleshooting and remediation of network issues, as well as improved correlation between application performance degradation and network bottlenecks. By providing a single-pane view of the network, Datadog's Network Path and SD-WAN monitoring solutions enable teams to quickly identify and resolve issues across cloud, WAN, and on-premises environments.

Datadog has introduced Sensitive Data Scanner (SDS), an agentless tool that scans Amazon S3 buckets and RDS instances for sensitive data, such as credit card numbers and personally identifiable information. SDS automatically pinpoints sensitive data in cloud resources, detects new resources, and flags potential security issues. The tool provides insights into the types of sensitive data found, prioritizes matches based on urgency, and offers remediation options to fix security issues. To access SDS, users must have Sensitive Data Scanner enabled and Amazon S3 buckets or RDS instances in their cloud environment, with optional 14-day free trial for new Datadog accounts.

Amber Bennoui and Matt Mills discuss the growing popularity of serverless infrastructure and how customers are deploying containers via AWS Fargate, which can introduce security challenges. Datadog Cloud Security Management (CSM) provides real-time threat detection for these environments, monitoring suspicious process and file activity, detecting anomalies like malicious file changes, and helping customers meet compliance requirements. The CSM solution is designed to detect threats targeting ECS and EKS containers deployed via AWS Fargate, including common tactics used by attackers such as exploiting vulnerable software, compromising AWS credentials, and deploying cryptominers. Datadog's security suite offers full-spectrum threat detection for these environments, providing coverage for attackers using compromised account credentials, malicious Docker images, and other techniques to gain initial access or execute on ECS containers. The solution is designed to surface threats in ECS and EKS workloads deployed via AWS Fargate today.

The open Kubernetes ecosystem provides several powerful tools to help manage resources effectively, including the Horizontal Pod Autoscaler and Vertical Pod Autoscaler, but implementing these tools requires significant effort from platform and application teams. Datadog Kubernetes Autoscaling provides multi-dimensional workload scaling recommendations and automation, enabling teams to deliver cost savings while maintaining performance and stability. This tool helps prioritize clusters and workloads for optimization by surfacing idle resources and providing time-series graphs of recent cost trends. Once a cluster is targeted, the tool enables teams to rightsize workloads directly within Datadog, providing complete recommendations and the ability to take action automatically or enable automation. By tracking progress over time, teams can track efficiency gains and idle cost savings, making it easier to optimize their Kubernetes clusters and save costs without sacrificing performance.

Datadog has introduced various features to help organizations improve their software delivery velocity, stability, and reliability. The company now collects and visualizes DORA metrics to track deployment frequency, lead time for changes, change failure rate, and time to restore service. Datadog also provides an out-of-the-box SLO dashboard that supports aggregated views of Service Level Objectives (SLOs) to help engineering leadership understand their organization's reliability at a glance. Additionally, the company has introduced CoTerm, which enables teams to livestream, record, and log terminal sessions for transparency into incident investigations. Furthermore, Datadog Sheets allows teams to analyze and share data using native spreadsheet functionality. The platform also provides Data Access Controls to protect sensitive information, while the Domain Allowlist enables customers to control which domains receive Datadog email notifications. These features aim to break down silos that separate teams and provide a unified view into an organization's environment.

At DASH 2024, Datadog announced a range of new products and features designed to enhance observability, security, and operational efficiency for organizations. Key highlights include the general availability of LLM Observability for monitoring generative AI applications, which allows for detailed tracing and error diagnosis. The Datadog Agent now incorporates an embedded OpenTelemetry Collector, streamlining integration and management. Additionally, Datadog introduced Log Workspaces for advanced log analysis, Live Debugging for efficient bug fixing, and Product Analytics for data-driven UX insights. In security, Agentless Scanning and Data Security help detect vulnerabilities and secure sensitive cloud data, while one-click infrastructure-as-code remediation and Datadog Code Security facilitate swift vulnerability management. For Kubernetes, Datadog offers Autoscaling for optimal workload management, and new features like Bits AI for autonomous incident investigation and Datadog On-Call for streamlined incident response enhance operational capabilities. These innovations aim to provide comprehensive solutions for monitoring, securing, and optimizing cloud environments.

The text discusses the benefits and risks of utilizing open source code in cloud-native applications, emphasizing the need for careful consideration due to potential security vulnerabilities and other risks such as malware, licensing issues, and unmaintained libraries. It highlights the importance of following best practices in open source management, including thorough evaluation of projects, staying updated on vulnerabilities, and adopting processes to manage and remediate security issues. The text advocates for the use of automation tools like Datadog Software Composition Analysis (SCA) to efficiently manage these risks, providing visibility into the software supply chain and enabling continuous evaluation. Datadog SCA enhances security by integrating real-time visibility into deployed services, thereby improving the security posture of organizations through collaboration across teams, including DevOps and SREs.

Datadog has streamlined incident troubleshooting by making critical infrastructure information accessible from within dashboards and monitor status pages. This allows responders to easily access detailed telemetry, monitor insights, and configuration changes, enabling them to identify probable root causes of infrastructure issues and take action to remediate them sooner. The new Recent Changes tab provides a 7-day history of configuration changes for any resource in the Resource Catalog inventory, allowing responders to quickly assess what change may have caused an incident. With this feature, responders can reduce the time needed to identify and resolve issues, making it easier to manage infrastructure and improve operational efficiency.

Datadog has enhanced its platform to facilitate more efficient incident troubleshooting by integrating critical infrastructure information directly into dashboards and monitor status pages, which are common starting points in investigative workflows. This integration allows users to open a Resource side panel from any timeseries graph widget, providing access to detailed telemetry, monitor insights, and configuration changes via a new Resource Changes tab. By consolidating these insights in a single view, Datadog enables users to more easily identify root causes of infrastructure issues and take corrective actions swiftly, thereby reducing the mean time to resolution (MTTR), minimizing operational stress, and improving business outcomes. For instance, in a scenario involving elevated HTTP 500 errors from an Elastic Load Balancer, users can quickly trace the issue back to a specific EC2 instance and identify that a recent IAM role change restricted its access to an S3 bucket. The platform's new features streamline the troubleshooting process by allowing users to track and roll back configuration changes efficiently, with the potential to resolve incidents in a timely manner, thus ensuring infrastructure stability and performance.

Fionce Siow and Ryan Warrier discuss the challenges of troubleshooting issues in data pipelines using engines like Apache Spark and managed platforms like Databricks or Amazon EMR. The main challenge is that these systems process large volumes of data in parallel, making it difficult to manually correlate relevant information from logs, infrastructure metrics, and job performance to find the root cause of failures. Datadog Data Jobs Monitoring (DJM) helps solve this problem by enabling teams to quickly detect and debug failing or long-running jobs while offering insights into job cost and optimization opportunities. DJM provides a unified view of all Spark and Databricks jobs and clusters across accounts and environments, allowing teams to identify issues with their data processing workloads and dive deeper to troubleshoot without relying on manual processes. It also enables teams to pinpoint and resolve job issues faster, reduce costs by optimizing overprovisioned clusters and inefficient jobs, and get a full view of how their data processing infrastructure is performing.

The Datadog Continuous Profiler now includes a timeline view that provides a detailed chronological visualization of code and runtime activity, allowing software engineers to pinpoint the root causes of performance issues in their applications. This feature offers a complementary perspective to traditional flame graph views, enabling users to identify causal factors behind runtime performance issues more easily. The timeline view helps engineers diagnose performance anomalies by providing visibility into thread-level activity, allowing them to detect non-parallelization and optimize code for better throughput, reduce p99 latency, observe code-level activity within spans, and analyze the impact of garbage collection on application performance in production. By using this feature, users can shed light on runtime issues that are otherwise hard to diagnose, fine-tune their services for optimal performance, and more quickly determine whether issues originate from the underlying infrastructure, runtime activity, or the code itself.

With the introduction of Google Cloud Actions, Datadog enables users to automate tasks such as scaling GKE node pools, blocking malicious IPs with Google Cloud Armor, and managing Google Compute Engine instances. This allows for more efficient management of infrastructure directly from within Datadog, reducing manual intervention and potential downtime. Automated workflows can be set up to scale critical services in response to alerts, freeing up teams to focus on strategic tasks rather than firefighting scaling issues. Additionally, App Builder provides a centralized platform for managing Google Cloud resources, enabling quick decisions based on real-time visibility into instance status and improving security posture by rapidly terminating or restarting instances associated with potential threats.

Datadog's App Builder is a low-code solution that enables teams to create custom apps within the Datadog platform, facilitating collaboration and direct action. These apps integrate natively into Datadog's monitoring platform, providing interactive visibility into telemetry data. By using a custom runbook app built with App Builder, teams can accelerate remediation by centralizing context and action into one unified view in Datadog, enabling them to identify issues, take immediate action, and communicate internally and inform customers about the status of their systems. The app builder allows users to create apps that integrate with various services and platforms, such as GitLab, Opsgenie, and Statuspage, and can be used to build a wide range of self-service tools to streamline DevOps processes, including database consoles, custom analyses, visualizations, and portals for developers.

Datadog's Continuous Profiler aims to make code profiling more accessible to engineers of all levels by introducing source code previews that contextualize profiling data, linking individual cells in flame graphs to relevant source code. To enable this feature, users need to set up Datadog's source code integration with their Git repositories and then hover over a flame graph cell to reveal the associated code. The preview also allows for more granular insights by grouping code lines instead of just methods, making it easier for engineers to troubleshoot performance issues in their applications.

Datadog has announced the expansion of its Ambassador Program, which recognizes and highlights individuals who contribute to the community through talks, courses, and blog posts. The program is doubling in size this year, with nine new ambassadors joining the existing group. These ambassadors include experts such as Apostolis Apostolidis, a software engineering practitioner, Ashley Parks, a senior DevOps engineer, Carles Javierre, an entrepreneur who helps businesses improve their infrastructure, Changhyeon Yoon, a frontend and DevOps engineer, Ibukun Itimi, an engineering manager, Santiago Gómez Sáez, a lead cloud architect, Kano Ichiro, the leader of Cloud Managed Services team at Toshiba Digital Solutions, Juliano Marcos Martins, a passionate tech leader and coder, and Anatoly Mikhaylov, a senior staff reliability engineer. The ambassadors have shared their experiences using Datadog and helped to foster the community through various activities such as speaking at events, developing courses, and writing blog posts.

The Secure by Design approach aims to tackle the challenges of developing and scaling secure applications. This involves considering potential security risks earlier in the development cycle, adopting a defense-in-depth strategy, and implementing multiple layers of security throughout the development process. To achieve this, organizations can follow principles such as a decentralized model for workstreams, a scalable system for sharing knowledge, and a customer-centric design for software. By doing so, teams can significantly reduce both the costs of fixing vulnerabilities and the risk of introducing them to customers. The benefits of adopting Secure by Design principles include cultivating a DevSecOps culture, improving overall developer experience and productivity, and proactively discovering and mitigating critical vulnerabilities in applications and services.

AWS Batch on Fargate is an AWS offering that combines the benefits of serverless compute engine for deploying and managing containers with a fully managed service for running batch workloads. It provides a cost-effective and scalable solution for running batch computing workloads without needing to manage underlying infrastructure. With the support for multi-container jobs now generally available, users can run compute-intensive tasks on serverless containers, making it ideal for machine learning, data processing, scientific computing, automated job scheduling, and serverless workflows. The Datadog Agent container enables comprehensive monitoring of containerized applications and jobs, collecting real-time metrics, traces, and live processes to ensure the health and performance of workloads. This includes monitoring CPU, memory, disk I/O, and network metrics, as well as tracing service calls and visualizing application architecture with a Service Map. Additionally, Datadog Live Processes allows users to monitor resource metrics like CPU and memory usage, isolate problematic processes, and detect anomalous behavior, helping to quickly troubleshoot and resolve performance bottlenecks.

Snowflake is an AI data cloud platform that enables organizations to break down silos and collaborate more effectively with partners and customers for storing, managing, and analyzing data. With Snowpark and Snowpark Container Services, developers can build applications and pipelines using familiar programming languages like Python and Java without moving data across tools or platforms. The Datadog Snowflake integration allows developers and data engineers to observe and act on their applications and data pipelines through Snowsight or third-party tools, leveraging metrics, traces, logs, and notifications as telemetry. With the new integration, Snowpark developers can ingest logs from all deployments, accounts, and regions into a single Datadog account, capturing logs and events in the Event Table and accessing them in Datadog via the new integration and standard logging and metrics. This enables quick action on Snowpark bottlenecks and failures by using Log Explorer to search for specific events or logs, creating custom monitors to alert on issues, and visualizing performance with Datadog's 850+ integrations.

DNS (Domain Name System) is a crucial component of network communication, facilitating the translation of domain names into IP addresses. DNS logs can provide valuable insights into network health and security issues, such as timeouts, DNS resolution errors, and malicious activity. Understanding the anatomy of a DNS log is essential to extract relevant information and identify potential security threats. By monitoring DNS logs, organizations can troubleshoot network connectivity and security issues, mitigate DNS attacks, and improve their overall network resilience. Datadog provides an integrated platform for monitoring DNS logs, offering features such as Logging without Limits, Pattern Inspector, and out-of-the-box dashboards for various DNS providers.

Datadog's AWS WAF integration enables organizations to monitor and analyze web ACL activity, metrics, and logs in one place. This provides a comprehensive view of security threats at the perimeter, APIs, and services within a network. Datadog's Application Security Management (ASM) complements existing protection by offering built-in threat intelligence, distributed in-app WAFs, and advanced log management capabilities. By leveraging Datadog's AWS WAF integration, organizations can extend their firewall protection to other layers of their environment, building a defense-in-depth strategy that includes perimeter firewalls, application security, and exploit prevention. This comprehensive approach enables better visibility into security threats, improved incident response, and enhanced overall security posture.

AWS provides built-in monitoring, logging, and auditing tools through Amazon CloudWatch and AWS CloudTrail to monitor and analyze AWS WAF metrics, activity logs, and audit logs. CloudWatch offers a high-level overview of web ACL activity, granular visibility into WAF metrics based on dimensions, and the ability to query logs using Logs Insights. Activity logs capture detailed information about traffic flowing through web ACLs, while audit logs provide insights into administrative activity for web ACLs. The AWS CLI enables users to query public APIs for information about their services, including metric data, allowing for automation of cloud infrastructure management. Amazon CloudWatch offers a central location for monitoring AWS WAF activity and tuning configurations, but may struggle with adapting to a constantly growing environment.

AWS WAF is a managed web application firewall that monitors network traffic to AWS applications and resources, providing perimeter-based security. It generates standard request metrics, CAPTCHA and challenge metrics, and bot control metrics to help monitor its performance and identify potential issues. Activity logs capture information about requests processed by web ACLs, while audit logs record any activity associated with user updates or access to web ACLs. Monitoring these logs can provide valuable insights into AWS WAF's efficiency and help ensure application security.

The Datadog mobile app provides deep visibility into applications and infrastructure, facilitating alerting, coordination among teams, and investigation of issues from anywhere. The app allows users to quickly set up for continuous visibility by logging in via QR code and customizing the home page layout for fast access to key resources. Users can also add widgets to their home screen for even faster access to important data, such as top-level health and performance metrics, monitor statuses, and timeseries data. The app enables users to take action on issues by troubleshooting, declaring incidents, and coordinating with teams, using features like the Log Explorer, Bits AI copilot, and search functionality. With uninterrupted visibility and quick action capabilities, the Datadog mobile app helps ensure continuous monitoring of system health and rapid response to emerging issues.

We unified our disparate software delivery processes by using Datadog's Service Catalog and Conductor to define the end-to-end delivery pipeline for an application, track pipeline speeds and developer experience using CI Visibility, monitor deployment progress and impact using CD Visibility and DORA metrics, providing a single pane of glass for end-to-end software delivery and minimizing context switching and leaky abstractions.