March 2024 Summaries

Organizations are increasingly adopting Google Cloud due to the rise of AI computing and a shift toward multi-cloud usage, which has led to a reconsideration of cloud security measures. To address this, Datadog has expanded its security platform to complement Google Cloud's built-in security features, enhancing visibility into potential threats and ensuring continuous compliance with industry standards like PCI DSS, HIPAA, CIS, and GDPR. By integrating with Google Cloud Security Command Center and Google Cloud Armor, Datadog allows security and operations teams to share event data, thus facilitating the identification and mitigation of vulnerabilities across APIs, microservices, and applications. This collaboration helps maintain an organization's security posture by providing insights into suspicious activities and ensuring compliance, especially as organizations continue to adopt serverless environments and expand their cloud infrastructure.

Monitoring your CI/CD system is crucial to optimize workflows and create best practices that minimize pipeline slowdowns and respond to CI issues. To achieve this, platform teams must collaborate with development teams to foster effective relationships and cultivate shared visibility into tests and CI pipelines. By configuring self-service tools that notify developers of new flaky tests and test performance regressions, platform teams can help developers identify, investigate, and remediate software testing issues. This includes creating dashboards that aggregate key test metrics, such as Datadog's out-of-the-box (OOTB) test visibility dashboard, which helps teams identify trends in flaky tests, performance regressions, and failing/slow test suites. Additionally, platform engineers can configure monitors that automatically detect new test issues in real-time, notify the appropriate individuals or teams about new flaky tests, test failures, and performance regressions, and provide developers with tools to track long-term trends in their software tests and identify new issues. By providing these visibility tools, both platform and development teams can analyze testing trends across their organization's repositories, gain quick insights into test executions, and make data-driven decisions to improve the reliability and performance of their CI/CD systems.

Austin Lai sat down with Marie-Laure Bardonnet, a Senior Engineering Manager leading engineering for Datadog's Log Management team. Marie-Laure has been at Datadog since 2017 and started on the Dashboards team in Paris before transitioning to the Logs backend team. She found the environment to be ideal for growth and learning new challenges. As an individual contributor, she then became a team lead and later an Engineering Manager II, managing all of the backend and frontend Logs teams. Marie-Laure's day-to-day involves crafting objectives and key results with her team, maintaining a balance between product and technical roadmaps, and helping people grow in their roles. She believes that career growth is about finding a balance between doing work you love, learning new things, and growing as a person. Datadog's culture has remained consistent despite scaling to over 5,000 employees, with pillars including the people and postmortem mindset. Marie-Laure's upbringing influenced her engineering background through preparatory classes in France, which instilled core values such as autonomy and critical thinking. Outside of work, she enjoys spending time with her husband and daughter, who was born after she joined Datadog. She advises her past self to be ambitious, ask questions, and overcome impostor syndrome, and encourages current interns and college students to show up, do the job, learn as they go, and grow.

The text outlines the enhancements to Datadog's Azure integration configuration page, which consolidates critical information about Azure integrations in a single view. This update aims to facilitate the assessment and adjustment of Azure monitoring strategies by allowing users to review and update configurations for app registrations and subscriptions from one location. It describes how users can manage telemetry data collection, address issues like expired client secrets, and monitor subscription activity crucial for security and cost management. Additionally, the Integrations tab offers insight into the metrics and statuses of various Azure integrations, helping organizations ensure comprehensive monitoring coverage and troubleshoot data collection issues efficiently. The enhanced features are designed to support seamless monitoring and observability for users leveraging Microsoft's cloud services through Datadog.

Datadog Mobile Application Testing provides fast, no-code, and reliable mobile app testing on real devices in the cloud. It allows teams to create and maintain automated tests that seamlessly integrate into CI/CD pipelines, ensuring high-quality releases. The platform supports end-to-end tests for iOS and Android applications, running them on real phones and tablets to provide realistic results. Datadog Mobile Application Testing also offers features like test recording, advanced options, assertions, subtests, local and global variables, step replayer, and comprehensive test result views. It complements other Datadog tools such as Mobile Session Replay, RUM, APM, and browser testing, offering a unified testing platform for all application needs.

The text discusses the importance and implementation of a service catalog for organizations to effectively manage and scale their systems. It emphasizes that a service catalog enriches services with ownership, configuration metadata, and monitoring data, creating a central knowledge base for service health, activity, and security. The process involves forming accurate service definitions, integrating relevant monitoring data, and ensuring consistent updates as services evolve. The text highlights the role of Datadog Software Catalog in supporting these practices by enabling service discovery, metadata enrichment, and governance through scorecards and integration with various monitoring tools. This approach facilitates streamlined incident response, service upgrades, and compliance audits across distributed architectures.

Cloud environments are vulnerable to various cyberattacks, including unauthorized cryptomining. Detecting these attacks is challenging as they often mimic regular cloud resource usage. Datadog CSM Threats helps identify and stop such activity by monitoring for suspicious processes and automatically killing them upon detection. This enables teams to focus on addressing security vulnerabilities and preventing future attacks. Key signs of unauthorized cryptomining include high CPU usage, overclocking, declining service performance, and sudden cost spikes. To prevent these attacks, teams should patch affected resources with the latest updates, fix access controls, and ensure proper configuration of cloud workloads.

Datadog CSM Threats is a solution designed to detect and stop unauthorized cryptomining activity on cloud workloads, which can quickly drive up computing costs and affect application performance. Cryptomining attacks are challenging to detect due to their ability to distribute software across multiple resources and encrypt outbound connections to mining pools. Signs of cryptomining activity include consistently high CPU usage, CPU overclocking, gradual decline in service performance, sudden spikes in cloud computing costs, and newly deployed clusters on infrequently used regions. Datadog CSM Threats identifies and proactively stops unauthorized mining processes via its Agent and built-in rules, providing context for developers, SREs, and dedicated security teams to confirm the process is no longer running, quarantine affected resources, and fix vulnerabilities or misconfigurations. To prevent future cryptomining attacks, teams can patch affected resources with the latest software updates, secure workloads by fixing overly permissive access controls, and configure resources with compliance best practices, which Datadog supports tracking across all cloud workloads.

Detecting malware in container environments poses significant challenges due to the rapid evolution of malicious code and the complexity of container stacks. Datadog Cloud Security Management (CSM) addresses these issues by providing a unified platform for malware detection that combines Datadog’s internal threat intelligence with third-party feeds like MalwareBazaar. This integration enhances precision and reduces false positives in identifying threats. CSM efficiently manages resource-intensive processes by executing them on Datadog's servers, allowing users to focus on analyzing and responding to security signals. The platform facilitates quick action by offering detailed insights into the detected malware, such as its location and impact, and provides tools to investigate further through various Datadog resources. As a result, CSM helps maintain the security of containerized environments by enabling rapid identification and containment of malware threats.

Cloud environments face significant security challenges due to their susceptibility to various cyberattacks, with unauthorized cryptomining emerging as a particularly difficult threat to detect and mitigate. This type of attack often disguises itself as regular system processes, leading to increased computing costs and diminished application performance before it is identified. Datadog Workload Protection addresses these issues by enabling proactive detection and cessation of cryptomining activities on cloud workloads, empowering DevOps and security teams to take control of their environments and facilitate the transition to DevSecOps. By leveraging the Datadog Agent and predefined rules, the system can automatically detect unauthorized mining processes and provide critical information to quarantine affected resources and rectify vulnerabilities. To prevent future attacks, it is crucial to patch vulnerable resources, secure access controls, and configure systems according to compliance best practices, all of which Datadog supports through its comprehensive monitoring tools.

Detecting malware in container environments is challenging due to the rapid evolution of malicious code and the complexity of container stacks, but Datadog Workload Protection offers a solution by providing a unified platform for malware detection. This platform leverages Datadog's internal threat intelligence and third-party feeds like MalwareBazaar to identify and remove threats with enhanced precision, using crowd-sourced threat data while minimizing false positives. The detection process, which is resource-intensive, is executed on Datadog's servers to prevent strain on users' resources. When malware is detected, Workload Protection generates a security signal, assigns a severity level, and provides detailed context and tools for users to quickly address the threat, including the ability to analyze related suspicious activities and infrastructure metrics. This robust approach allows users to secure their containers effectively without incurring significant computational overhead, and Datadog offers further resources and a free trial for new users.

Kata Containers is an open source project that enhances container security by isolating them in lightweight VMs. It integrates with common container management software and provides the same security advantages as VMs, making it suitable for sensitive financial workloads or multi-tenanted cloud environments. Datadog now supports Kata Containers, allowing users to visualize and alert on metrics, traces, and logs from their Kata Containers alongside other containerized environment data. To use Datadog with Kata Containers, install the latest version of the Agent and begin collecting telemetry through built-in containerd endpoints. This integration provides benefits such as automatic anomaly detection and easy troubleshooting for secure workloads in Kata Containers.

Wassim Dhif`, `Danny Driscoll`, and `JC Mackin` are part of the Kata Containers project, an open source initiative aimed at enhancing security for containers by isolating them in lightweight VMs. Kata Containers run with the speed and flexibility of standard containers, integrating easily with common container management software like Docker and Kubernetes. This isolation layer improves security but may impact observability platforms' ability to capture telemetry signals. Datadog has announced its support for Kata Containers, allowing users to visualize and alert on metrics, traces, and logs from these workloads alongside other containerized environments. To get started with Kata Containers, users can install the latest version of the Datadog Agent, which collects telemetry through built-in containerd endpoints, providing benefits like anomaly detection and data consolidation. The project offers a novel high-security option for containerized workloads while maintaining traditional container management ease, making it an attractive choice for sensitive or secure applications.

Debugging errors in production environments can be challenging and time-consuming without access to the inputs and associated states that caused the errors. Datadog's Exception Replay feature helps developers reproduce exceptions with real production state and inputs, allowing them to identify the root cause of bugs faster. By capturing local variable data and execution context of production errors in APM Error Tracking, it enables users to analyze the steps leading up to an error and obtain a contextual understanding of the environment in which the error occurred. Exception Replay is now available in public beta, providing developers with valuable tools for efficient troubleshooting and minimizing future errors.

Datadog's board of directors, led by Titi Cole and Julie Richardson, represent the interests of shareholders and provide guidance to the executive leadership on critical decisions. The two women bring their expertise in financial services and private equity to help Datadog serve its growing customer base, particularly in the financial services industry. They also support the company's workplace culture values, including diversity, inclusiveness, and a focus on delivering exceptional customer experiences. As public company leaders, they emphasize the importance of sustainability and scaling sustainably while maintaining quality product management, sales force effectiveness, and positive customer experiences.

Datadog's Exception Replay automatically captures local variable data and execution context of production errors in APM Error Tracking, enabling developers to quickly reproduce exceptions with real production state and inputs. This feature provides contextual understanding of the environment in which an error occurred, allowing developers to analyze the steps leading up to an error and obtain a better understanding of the system's behavior. By using local variable data, developers can accelerate debugging and simplify the process of identifying the root cause of errors. Exception Replay also enriches existing stack traces with production variables that triggered the error, making it easier to investigate and fix issues. Additionally, this feature provides execution context that helps developers understand their production state better, leading to more efficient troubleshooting, an improved user experience, and vital information for implementing preventative measures.

Container-native technologies offer increased scalability and speed of deployment in containerized infrastructure but also present new monitoring challenges. Datadog's growing suite of container-native technology integrations enables users to monitor their entire containerized infrastructure from one place, ensuring consistent application performance and maintaining an exceptional end-user experience. These integrations cover various tools such as service meshes with Istio and Envoy, autoscaling and resource utilization with Karpenter, continuous integration and continuous delivery (CI/CD) with Flux, and messaging and streaming with Strimzi. Datadog's out-of-the-box dashboards provide a high-level view of the health and performance of these tools, allowing users to make informed decisions on how to optimize their containerized environments.

The Sankey visualization page in Datadog Product Analytics helps users identify the most popular paths taken by users within their application. This tool provides high-level overviews of common user journeys, allowing businesses to find key journeys for further analysis. By using Sankey diagrams, companies can determine which routes are the most accessible and understand why users choose certain paths. The Sankey visualization also enables users to create funnels directly from the diagram or pivot to Datadog Real User Monitoring (RUM) for more in-depth analyses on factors driving user choices.

Datadog has a comprehensive suite of integrations with container-native technologies to monitor the entire containerized infrastructure from one place. This includes service meshes like Istio, Envoy, and Traefik, security and compliance tools such as Kyverno, autoscaling and resource utilization solutions like Karpenter, and software delivery automation tools like Flux, Argo, and Tekton. Additionally, Datadog integrates with messaging platforms like Strimzi to provide visibility into the performance of containerized applications. With these integrations, teams can gain real-time insights into their container-native technology stack, ensuring consistent application performance and a unified view of their mesh's performance across their containerized environment.

The text discusses how Datadog Product Analytics' Pathways diagrams can enhance the analysis of user journeys within an app by providing high-level overviews of common paths users take. These diagrams help identify key user journeys and facilitate the creation of customized visualizations to study these paths in detail, particularly when trying to optimize user experience (UX). By examining these pathways, developers can determine which routes are most accessible and why, whether due to intuitive navigation or visually pleasing design. Once key journeys are identified, funnels can be created directly from Pathways diagrams to analyze user behavior further. The integration with Datadog Real User Monitoring (RUM) allows for a deeper understanding of the influence of design or performance on user choices. Pathways enable UX designers to adjust their strategies, such as modifying homepage banners or enhancing popular paths, to better guide users and improve overall satisfaction with the app.

Developers often face challenges in troubleshooting issues between development, testing, and production environments due to variables such as large-scale user requests and increased server load. Datadog offers plugins for various IDEs like VS Code, Visual Studio, IntelliJ IDEA, GoLand, and PyCharm to provide visibility into live production data directly within the coding environment. These plugins help developers correlate security issues, high resource consumption, and degraded performance directly to the methods and lines of code responsible without leaving their coding environment. This reduces context switching and creates a more streamlined investigative workflow. The IDE plugins can be used for highlighting performance, reliability, and security issues; identifying resource-intensive methods; and proactively fixing noncompliant code prior to execution.

Datadog offers plugins for popular IDEs like VS Code, Visual Studio, IntelliJ IDEA, and PyCharm to provide developers with visibility into live production data directly within their coding environment. This allows them to identify performance, error, and reliability issues, fix noncompliant code prior to execution, and troubleshoot resource-intensive methods without leaving their coding environment. The plugins provide features such as Code Insights, Continuous Profiler, and Static Analysis to help developers streamline their investigative workflow and improve the overall velocity of their development process. With these tools, developers can reduce context switching, gain insights into their application's performance and security, and make data-driven decisions to optimize their code and improve their application's reliability.

As of August 2023, only three out of 23 US government agencies were compliant with Office of Management and Budget (OMB) requirements for log management and security observability. These requirements are outlined in M-21-31, a 2021 memorandum that was issued following Executive Order 14028 on improving national cybersecurity. Datadog offers Log Management, Cloud Workload Security, and Cloud SIEM in its US1-FED region on AWS GovCloud, aligning with our FedRAMP® Moderate Authorization. This enables seamless compliance for public sector organizations using Datadog. The solution brief explains how Datadog helps organizations address the challenges associated with M-21-31 compliance by meeting log retention and storage, log management and access, and security threat management requirements.

Datadog has developed solutions that align with the US government's Office of Management and Budget (OMB) requirements for log management and security observability, specifically M-21-31. The company offers Log Management, Cloud Workload Security, and Cloud SIEM in its US1-FED region on AWS GovCloud, which enables seamless compliance for public sector organizations using Datadog. These solutions help meet the requirements for log retention and storage, security threat management, investigate threats with historical context, detect emergent threats in real-time, and secure public sector applications. By providing a comprehensive cloud security solution, Datadog supports the nation's cybersecurity efforts and helps public sector organizations comply with federal requirements.