May 2025 Summaries

The Google Cloud Next event highlighted significant developments in Google Cloud's AI capabilities, storage solutions, and security enhancements, with Datadog playing a prominent role as a key partner. Google introduced innovations such as the Ironwood AI accelerators, Gemini 2.5 models, and the A2A protocol, while expanding AI integration in services like BigQuery to enhance data analytics. To support these advancements, Google announced new storage offerings to improve AI workload performance and introduced Google Unified Security to streamline security management. Datadog, a recipient of multiple Google Cloud Partner of the Year awards, showcased its tools for monitoring and optimizing Google Cloud environments, including AI observability, storage monitoring, and expanded BigQuery and security monitoring capabilities. These features aim to help organizations maximize performance, reduce costs, and enhance security across their Google Cloud infrastructure.

The text discusses the issue of hallucinations in large language models (LLMs), where the models generate false or unsupported information, potentially spreading misinformation and eroding user trust. It highlights retrieval-augmented generation (RAG) as a method to reduce such occurrences by providing LLMs with verified context; however, it doesn't completely prevent hallucinations. To tackle this, Datadog introduces a hallucination detection feature within its LLM Observability framework. This tool evaluates LLM-generated text against the provided context to identify discrepancies and automatically flags hallucinated responses, offering insights into their frequency and impact. The feature distinguishes between 'Contradictions' and 'Unsupported Claims,' allowing users to customize detection based on sensitivity levels. Additionally, the system provides tools to trace the source of hallucinations, analyze patterns across applications, and correlate them with various factors such as deployments or traffic changes, thereby enhancing the reliability and credibility of LLM outputs.

Datadog has introduced the Wildcard widget, a new tool designed to offer users enhanced customization options for visualizing data within their dashboards. Utilizing the open-source Vega-Lite grammar of graphics, the Wildcard widget allows users to create bespoke, code-defined visualizations that cater to specific use cases, such as unconventional data formats or external data sources. This flexibility enables users to craft detailed visual representations, including categorical heatmaps and interactive geomaps, by leveraging a powerful query editor that supports complex data transformations and enriches visualizations with external data. The widget's integration with existing dashboards facilitates the creation of expressive, layered charts, allowing users to analyze data in innovative ways that reveal insights beyond standard visuals. With built-in tools for previewing and refining data, the Wildcard widget streamlines the process of creating, editing, and fine-tuning visualizations, ultimately transforming complex data into clear and actionable insights.

The text delves into the advantages and applications of Datadog's nested metric queries, emphasizing their capability to enhance visibility and control over distributed applications by allowing users to perform complex queries on large datasets. These nested queries enable multilayer aggregation, facilitating the analysis of telemetry data across different levels of granularity, from entire infrastructures down to individual components. This flexibility supports various use cases, such as resource capacity planning, where accurate forecasting of CPU utilization over time is crucial, as well as load balancing in Kafka topics to identify and rectify imbalances. Furthermore, the use of nested queries allows for the calculation of percentiles on count, rate, and gauge metrics, aiding in resource provisioning and network monitoring, while also enabling the retention of data granularity over long-term analyses for executive reporting and resource allocation. The text concludes by encouraging readers to explore these functionalities through a 14-day free trial of Datadog.

Datadog is introducing Log Workspaces, a tool that helps organizations take control of their log data. The new feature extends the capabilities of the Datadog Log Explorer, providing a flexible and adaptable environment for in-depth analysis of log data. With Log Workspaces, teams can query, join, and transform log data from any number of sources using SQL, natural language, or visualizations. This enables virtually anyone in the organization to analyze log data, regardless of their level of experience. The tool also provides transformation cells that allow users to extract information from logs at query time, making it easier to add granularity and structure to the data. Log Workspaces can help teams accelerate incident response by making quick work of complex log analysis and incorporate fine-tuned log-driven datasets into their day-to-day monitoring.

The text outlines how integrating Datadog cost data with Cloud Cost Management (CCM) can enhance cost visibility and accountability for organizations scaling their Datadog footprint. By providing real-time, contextualized cost data tied to services and teams, Datadog allows users to allocate spending accurately, prevent cost overruns, and proactively manage expenses. The integration includes features such as CCM Explorer, dashboards, notebooks, and monitors that allow for an analysis of cost along with performance metrics, enabling quick assessment and troubleshooting. Custom tagging and usage attribution further help map costs to specific teams and services, while cost monitors alert teams to unexpected changes, reducing end-of-month surprises. This system enables organizations to understand their total cost of ownership by viewing Datadog costs alongside those from cloud and SaaS providers, thus facilitating comprehensive cost management across their infrastructure. The Datadog cost data is available to all CCM customers at no additional cost, with a 14-day free trial for new users.

The announcement introduces Toto, a state-of-the-art time series foundation model, and BOOM, a new public observability benchmark, both of which are open source and available under the Apache 2.0 license. Toto, trained solely on Datadog's internal telemetry metrics, excels in performance on the BOOM benchmark, and other established benchmarks like GIFT-Eval and LSF, by effectively handling the unique challenges posed by observability metrics such as sparsity, spikes, and high-cardinality multivariate series. The BOOM benchmark, which comprises 350 million observations across 2,807 real-world time series, is designed to evaluate the performance of models on observability metrics, which are crucial for operations like anomaly detection and predictive forecasting. Toto's architecture incorporates innovative features such as a Student-t mixture model prediction head and a patch-based causal normalization approach, allowing it to generalize well to nonstationary data and achieve superior performance. The availability of these tools aims to advance the field of time series analysis and encourage community engagement, with Datadog seeking to expand its AI research team.

The shared responsibility model, introduced by AWS in 2011, delineates the division of security responsibilities between cloud providers and customers, with the former securing physical infrastructure and the latter securing data, configurations, and access. As cloud environments have evolved, the complexity and variety of services have increased, leading to more challenges for customers in meeting their responsibilities, thereby heightening security risks due to potential misconfigurations. This evolving landscape necessitates collaboration between security and engineering teams to manage security responsibilities effectively, a practice exemplified by Datadog, which has integrated a shared responsibility approach within its organization. The model varies slightly among providers like AWS, Azure, and Google Cloud, each offering resources and tools to help customers adhere to their security duties. However, the overarching aim remains the same: to bridge the knowledge gap and foster a unified approach within organizations to enhance security posture and prevent breaches, emphasizing the importance of making security a collaborative effort across all teams involved.

The implementation of Datadog Real User Monitoring (RUM) has significantly enhanced the ability to understand and improve user experience by providing detailed insights into user interactions and performance data. Before RUM, the company relied on frontend logging, which offered limited visibility into user-centric issues. With RUM, they can now access unbiased feedback and analyze user behavior, which has led to tangible improvements in their UI, such as the development of Quick Graphs and an enhanced understanding of feature adoption. Additionally, RUM has facilitated frontend modernization by informing browser support strategies and optimizing UI across various screen sizes. The use of Session Replay has further allowed the team to troubleshoot issues effectively by providing an unfiltered view of user journeys. By leveraging these insights, engineering decisions are more data-informed, leading to features that positively impact customer workflows. The company emphasizes a "data-informed" approach, utilizing customer feedback through direct conversations, product research tools, and design partnerships, while mitigating bias through RUM's unbiased data collection. Overall, RUM has empowered the organization to prioritize user-centric enhancements, ensuring a seamless and efficient user experience across devices.

Businesses increasingly rely on LLM proxies to streamline the integration and governance of large language models, offering a centralized interface that simplifies model access and ensures compliance. However, these proxies introduce monitoring and visibility challenges, such as difficulties in tracing issues back to either the model or the proxy logic, and potential security vulnerabilities from mishandling sensitive data. Effective monitoring is crucial to avoid performance risks, such as blocking valid prompts or misrouting requests, which can degrade application performance. Tools like Datadog LLM Observability provide end-to-end visibility and trace-level insights, allowing teams to optimize model routing, control costs, and maintain performance by tracking model usage and identifying expensive behaviors. By doing so, teams can adjust routing rules to prioritize cost-effective models and set alerts to prevent unexpected overages, ensuring efficient and cost-effective LLM usage.

The Datadog Developer Hub is a new centralized platform designed to help developers more easily find and utilize the integrations, libraries, and tools necessary to customize and extend Datadog's monitoring and observability solutions. This hub consolidates resources that were previously dispersed across various repositories and documentation pages, offering a searchable catalog of both official Datadog and community-contributed integrations, as well as various libraries, sample apps, and learning resources. The Developer Hub also encourages community engagement, inviting developers to contribute and join a growing network of users dedicated to enhancing Datadog's capabilities. The platform aims to continuously expand, providing a comprehensive destination for developers to optimize their observability workflows, and offers a 14-day free trial for new users interested in exploring its features.

Kotlin Multiplatform is increasingly being used by mobile developers to share business logic across iOS and Android, though it often creates challenges in visibility and monitoring. The Datadog Kotlin Multiplatform SDK addresses these issues by providing a unified integration that enhances observability in shared code, offering tools like Real User Monitoring (RUM), crash reporting, error tracking, session replay, and logging capabilities. This setup allows mobile teams to debug performance issues, investigate crashes, and establish a single source of truth for cross-platform monitoring, thereby reducing blind spots and improving collaboration. By using Datadog's tools, developers can track app performance, stability, and user behavior consistently across platforms, facilitating faster issue resolution and more cohesive user experiences.

Amazon Simple Email Service (Amazon SES) is a critical tool for organizations to send transactional, marketing, and newsletter emails, but it has become a common target for phishing campaigns and cloud control plane attacks. Attackers often exploit misconfigured SES accounts by using compromised access keys to make API calls that assess account availability, potentially leading to phishing campaigns that damage an organization's reputation. Common vulnerabilities include inefficient email logging, lack of email authentication, and overly broad IAM permissions, all of which can provide entry points for attackers. Monitoring email volume, bounce rates, and complaint rates can help identify anomalies indicative of phishing activity. AWS CloudTrail logs and Datadog's monitoring tools can be used to trace account changes and detect suspicious behavior. Strengthening SES security involves addressing configuration risks and monitoring crucial SES metrics to prevent unauthorized access and mitigate potential threats.

The text discusses how Datadog DORA Metrics can help teams improve their software delivery performance by providing detailed insights and analysis through key metrics like deployment frequency, change lead time, change failure rate, and time to restore service. By automatically collecting data from various sources, including CI/CD pipelines and incident management workflows, Datadog enables teams to identify bottlenecks, analyze delivery workflows, and launch targeted initiatives to optimize their development lifecycle. It emphasizes the importance of understanding and acting on DORA metrics to drive meaningful improvements, such as reducing approval bottlenecks or enhancing team experimentation. The tool also integrates with other observability data to correlate delivery performance with system health and business metrics, allowing teams to make informed decisions and ensure long-term success. Datadog DORA Metrics is available for free to Datadog customers, offering a holistic view that connects software delivery patterns to real-world outcomes.

The introduction of new FinOps capabilities in Datadog Cloud Cost Management (CCM) offers a unified platform that integrates cost data and observability data to enhance collaboration between engineering and FinOps teams. This integration allows for more effective cost allocation, budget tracking, and anomaly detection, enabling organizations to optimize cloud and SaaS expenditures. Custom allocation rules, cost reporting, and budget visibility are provided to help engineering teams monitor spending in real-time, preventing cost overruns. Anomaly detection is powered by machine learning models, facilitating quick identification and resolution of unexpected cost changes. Additionally, CCM offers features like cost recommendations and commitment programs to help teams optimize both usage and rates. Overall, the platform aims to streamline workflows, improve cost efficiency, and foster a culture of cost ownership across organizations.

The text discusses the features and benefits of the Datadog Agent for monitoring and managing infrastructure and applications. The Agent enhances visibility by collecting high-frequency, high-resolution telemetry data, enabling rapid troubleshooting and incident response. It supports custom metrics and correlates telemetry across different data types, facilitating more effective monitoring and analysis. The installation and management of the Agent are streamlined through Datadog Fleet Automation, which provides centralized control and configuration across diverse environments. The Agent ensures data security with compliance to industry standards and encrypts data transmissions. For organizations using OpenTelemetry, the Datadog Distribution of the OTel Collector offers expanded support, enhancing interoperability. The text encourages users to leverage these capabilities for improved system observability and offers a free trial for new users.

Datadog Cloud SIEM is designed to provide real-time visibility into security events across cloud-native environments, enabling organizations to protect themselves from legitimate threats. It is deeply integrated with Datadog Log Management and Flex Logs, offering comprehensive and cost-effective security insights. The platform enables real-time and historical analysis on logs, powering advanced security monitoring, threat detection, and compliance. Cloud SIEM also offers a historical engine, which enables users to conduct investigations on past logs and understand the context of previous security incidents. Additionally, Datadog Cloud SIEM simplifies the process of onboarding security teams with Content Packs, which are curated lists of integrations that collect valuable security logs. The platform can be easily integrated with existing log management systems, such as Flex Logs, to provide end-to-end coverage for detecting, investigating, and resolving security issues.

Datadog offers an integration with Airflow that enables the ingestion of Airflow metrics into Datadog, allowing users to visualize and filter these metrics in a single consolidated view. This integration includes features such as alerting on key Airflow metrics, monitoring Airflow logs, and providing Data Jobs Monitoring for proactive detection and troubleshooting of issues within Airflow pipelines. Additionally, Datadog's Log Management feature collects Airflow logs and displays them within the Log Explorer, while Data Jobs Monitoring aggregates live data on each task execution for Airflow DAG runs, represented as Datadog traces. This integration provides comprehensive visibility into Airflow workflows and supports a single source of truth for monitoring data pipelines alongside other technologies supporting applications.

The Datadog User Groups program aims to formalize support and recognition for local communities that bring together fellow users, share knowledge, and collaborate. Inspired by the success of existing groups in Japan, Korea, and Germany, the program will enhance the experience through collaborations with community organizers, content development, and support. The initiative seeks to enable the creation of additional user groups, particularly in Paris and New York City, and is already spotlighting successful community leaders who have led the way by bringing users together globally. These groups have reported success in sharing insights about Datadog, covering integrations, best practices, and case studies related to DevOps, observability, and security, ultimately helping members use their spending more effectively and operate robust services.

Airflow allows users to collect metrics, logs, and traces from their pipelines using its native tooling. Users can customize the ingestion of metrics by specifying categories such as scheduler, executor, dagrun, pool, triggerer, and celery. This helps remove unwanted data, reduce noise, and save on intake costs. Airflow generates component logs automatically, which are mostly useful for pre-production testing and debugging. Scheduler logs contain critical information about task queue performance and runtime events, while worker logs capture information about the runtimes of worker processes as tasks are submitted, run, and cleaned up. Task logs record data for specific DAG runs, enabling users to troubleshoot failed or retried task instances. Airflow includes an OpenLineage provider that enables users to send lineage events for task executions to record run metadata and job metadata. Users can monitor metrics, logs, and traces using Airflow's native webserver interface, including views such as the Cluster Activity View, Grid View, Graph View, and Gantt Chart. Additionally, users can use external tools like FluentD and Marquez to collect and visualize log data.

The Datadog Ambassadors community has grown significantly over the past year, with a new cohort of builders, security experts, educators, and community leaders joining the program. The ambassadors have been actively engaged in sharing their expertise through various channels such as blog posts, conference talks, open source contributions, and community leadership, focusing on topics like observability, DevOps, security, and performance optimization. Many ambassadors have taken on leadership roles within their local communities, scaling User Groups into thriving hubs of learning and collaboration, and fostering strong regional ecosystems that support ongoing learning and connection. The program continues to expand with the addition of new ambassadors, who bring diverse expertise and passion to the community, shaping how teams around the world think about observability, reliability, and performance.

Airflow is a popular open source platform that enables users to author, schedule, and monitor workflows programmatically. It helps teams run complex pipelines by providing task orchestration, dependency management, and efficient scheduling across many different tools. Airflow is particularly useful for creating data processing pipelines, orchestrating machine learning training, and running cloud services. The platform is widely used in data processing use cases, with 85.5% of users relying on it for ETL/ELT pipelines. By defining workflows as Python code, teams can manage them more efficiently, automate creation, scheduling, and provisioning, and make them parameterizable and extensible. However, running workflows in production requires monitoring potential problems such as task failures and latency, task throttling, orphaned tasks, and more. Airflow's scheduler uses Directed Acyclic Graphs (DAGs) to define workflows, which can be parsed by the scheduler to schedule and run defined tasks. The scheduler is managed by a pool of workers that execute tasks, with the triggerer helping to defer worker slots to run new tasks while queued tasks are waiting to execute. Airflow offers a suite of out-of-the-box metrics that can be collected using StatsD to track key performance indicators such as task failures, operator failures, DAG run duration, schedule delay, number of queued tasks, and more. By monitoring these metrics, teams can identify potential issues, optimize their workflows, and ensure the reliability and performance of their Airflow deployments.

In the OpenTelemetry deployment strategy landscape, the no-collector pattern sends telemetry signals directly from applications to backends without using an OTel Collector. This approach provides minimal configuration and management overhead but tightly couples the application to the backend, leading to potential service interruptions if the backend becomes unavailable. In contrast, the agent collector deployment pattern introduces a single instance of the OTel Collector between the application and the backend, offering benefits such as a local cache for storing telemetry data and pipeline processors for enriching signals. The gateway collector deployment pattern introduces an additional layer of aggregation, consolidating data streams before pre-processing telemetry data or applying policies, making it suitable for large-scale environments with complex requirements. When considering which deployment strategy to use, factors such as the need for identical data pipelines, tail-based sampling, and processing pipelines should be taken into account.

Datadog has achieved "In Process" status for FedRAMP High authorization, marking another milestone in its commitment to providing the highest level of cloud security and observability for government agencies. This designation enables government IT leaders and engineers to prepare for full-stack observability at the highest federal compliance level. With Datadog's unified platform, government agencies can gain real-time insight into their hybrid and multi-cloud systems, detect anomalies, resolve issues, and meet evolving mandates like OMB M-21-31, EO 14144, and TIC 3.0. The platform supports ongoing efforts to modernize federal IT operations by providing a secure, unified platform that delivers more than just compliance—it powers better decision-making, operational agility, and measurable cost savings.

This Month in Datadog' is a monthly update of the company's latest features, product announcements, and more. The April episode covers tracking GitHub Copilot usage, avoiding mistyped terminal commands with Datadog CoTerm, and pinpointing browser performance issues. A new feature was introduced, unifying OpenTelemetry and Datadog with the distribution of the OpenTelemetry Collector, which offers an enterprise-ready, OTel-native Collector distribution for a unified experience. Additionally, features like tracking GitHub Copilot usage across organizations, avoiding mistyped terminal commands, and pinpointing browser performance issues were highlighted, along with new learning paths on the Datadog Learning Center and an in-depth preview of DASH 2025.

Datadog has introduced Network Device Monitoring (NDM) to monitor Cisco Meraki's entire line of corporate IT infrastructure devices, including wireless access points, switches, and network security appliances. With NDM, users can visualize the health of their wireless network, troubleshoot and alert on device issues, and monitor and take action on network and security event logs. The integration provides a unified tool to monitor Meraki devices alongside traditional SNMP-based network devices in Datadog Network Device Monitoring. It offers insights into access point utilization, channel performance, and security appliance performance, enabling users to identify potential performance issues, troubleshoot device problems, and prevent security incidents. The integration is available for free trials, allowing users to start monitoring their Meraki environment with comprehensive visibility into the health and performance of their network and infrastructure.

Azure SQL Managed Instance, a fully managed platform-as-a-service (PaaS) database engine, is gaining popularity due to its ability to offload routine management tasks such as upgrades, patches, backups, and monitoring to Microsoft, ensuring 99.99% uptime on high-availability infrastructure. This makes it an attractive option for organizations migrating from on-premises to cloud databases. Datadog's integration with SQL Managed Instance offers comprehensive monitoring, allowing users to optimize database usage and performance proactively. The integration provides a centralized dashboard for a high-level overview of SQL Managed Instance usage, performance, and inventory, including monitors for CPU utilization and insights into I/O and storage metrics. These features enable users to plan capacity effectively, optimize queries, and prevent performance degradation. Additionally, Datadog offers various integrations for Azure services and provides resources for monitoring and optimizing database performance, including a 14-day free trial for new users.

Michael Whetten, a developer and product team leader, highlights the challenges of measuring business impact in AI-powered applications. Without systematic experimentation and evaluation, teams are left making decisions in the dark, unable to confidently determine which improvements add value. Eppo, an experimentation and feature-management platform, is joining forces with Datadog to bring end-to-end observability, feature management, and experimentation into a single unified platform. This integration will enable teams to test changes, analyze user behavior, and measure business impact at scale, allowing them to move fast and ship with confidence. Eppo's capabilities will continue to support existing customers while bringing on new ones as part of the Datadog ecosystem.

The OpenTelemetry Collector (OTel Collector) is an open-source observability framework that provides a standardized approach to collecting, processing, and exporting telemetry data across distributed systems. The Datadog Distribution of the OTel Collector (DDOT Collector) enables users to adopt OTel flexibility while taking full advantage of Datadog's advanced features such as Live Container Monitoring, Network Monitoring, eBPF-based Universal Service Monitoring and Cloud Security Management Threats. With DDOT Collector, teams can extend the OTel Collector's feature set with broader and deeper observability, simplify OTel Collector management at scale with Fleet Automation, confidently accelerate OTel adoption with enterprise-grade reliability, security, and support, and unify their OpenTelemetry and Datadog experience today.

The Datadog Learning Center offers interactive courses and learning paths to help users get started with or learn more about different Datadog features and their use cases. The center has introduced new universal, persona-based, and product-based learning paths that cover foundational concepts, skills relevant to specific engineering personas, and skills directed to specific Datadog product areas and monitoring workflows. These learning paths are designed to help users quickly discover the courses that are most relevant to their needs, align their coursework with their learning goals, and prepare for a Datadog certification exam. The learning center also offers certification preparation learning paths, interactive course material, and a free 14-day trial for new users.

Datadog Error Tracking provides teams with a single source of truth for errors across frontend and backend systems, enabling quick identification and prioritization of critical errors, acceleration of troubleshooting with full-stack visibility, proactive detection and prevention of regressions, and efficient tracking and resolution of errors. The platform automatically groups similar errors based on type, message, and stack trace, assigns suspected cause labels, provides unminified stack traces with direct links to source code, offers real-time alerts for new or escalating issues, and integrates with the Datadog observability platform for better visibility and context.