July 2021 Summaries

Amazon FSx for Windows File Server is a fully managed file storage service built on Windows Server that enables organizations to reduce operational overhead and take advantage of the flexibility and scalability of the cloud. AWS recently added file access auditing to the Amazon FSx service, which publishes and stores audit event logs summarizing file system access activity at user-level for all files, folders, and file shares. Datadog can be used as an endpoint to send Amazon FSx audit event logs for retention and real-time analysis, allowing users to monitor access activity across their file systems, create security rules to alert them to possible threats, and analyze and monitor Amazon FSx audit event logs in Datadog.

Amazon FSx for Windows File Server is a fully managed file storage service built on Windows Server. Migrating to this service enables organizations to reduce operational overhead and take advantage of cloud flexibility and scalability, while maintaining visibility into file access activity, which is crucial for security and compliance requirements in sectors like financial services and healthcare. AWS recently added file access auditing to the Amazon FSx service, publishing and storing audit event logs that summarize user-level file system access activity at a file share level. Datadog can be used as an endpoint to send these logs for retention and real-time analysis, allowing teams to monitor access activity, create security rules to alert them to possible threats, analyze and monitor Amazon FSx audit event logs, automatically detect security threats to their FSx file system, and get deeper insight into their file systems.

Datadog has introduced new DNS monitoring features to help troubleshoot end-to-end DNS issues and ensure the performance and availability of applications. The DNS view in Network Performance Monitoring provides insights into internal DNS server health, service discovery, and Synthetic DNS tests for proactive detection of failures and misconfigurations. These features enable users to assess the health of all internal DNS servers, investigate DNS communication from the client side, troubleshoot latency and failure issues, correlate DNS performance with server monitoring data, and detect irregularities in DNS record mapping and resolution times. Datadog's unified visibility into DNS traffic helps users identify problems with individual DNS servers, correlate DNS traffic with server health, and proactively monitor their DNS records for potential issues.

Datadog's Session Replay feature allows developers and designers to gain visibility into real user behavior when navigating large customer-facing applications, helping optimize UX. By recording individual user sessions in a video-like interface, it enables reproducing bugs and understanding patterns in users' behavior. This tool can speed up debugging processes by saving time and eliminating guesswork in recreating bugs. It also helps UI/UX designers understand the efficacy of their designs by observing how users interact with websites. Additionally, Datadog offers privacy options to obfuscate sensitive user information during session replays.

Datadog's Session Replay feature allows developers to watch individual user sessions in a video-like interface, providing visibility into real user behavior and helping them optimize their UX. By reproducing bugs and understanding patterns in user behavior, developers can speed up their debugging process and improve the overall user experience. The feature also provides designers with valuable insights into how users interact with their application, enabling them to create design interventions that address common pain points and improve conversion rates. To mitigate customer privacy risks, Datadog offers default privacy options, including three levels of obfuscation for sensitive information. With Session Replay, developers can leverage qualitative context around frontend performance metrics, helping them iterate fixes faster and deliver a better user experience.

Datadog's Continuous Integration (CI) Visibility feature provides comprehensive visibility into CI/CD workflows, helping organizations identify issues with their builds and testing. It complements Datadog's existing integrations with popular CI providers and supports synthetic tests in CI pipelines. With CI Visibility, users can monitor pipeline builds, stages, and jobs to locate problems, track test performance, and identify flaky tests. The feature works with GitLab, GitHub Actions, Jenkins, CircleCI, and Buildkite. It automatically instruments pipelines and tests for end-to-end tracing without requiring manual reproduction of test failures. CI Visibility is now generally available to all Datadog customers.

This post discusses the use of private locations and testing tunnels in Datadog Synthetic Monitoring to test different types of applications that are not publicly available, such as local versions of production websites and internal applications for employees. The testing tunnel is a secure tunnel connection that requires little setup and can be used for on-demand testing in local and continuous integration (CI) environments. It leverages Datadog's command line interface (CLI) to create an end-to-end encrypted HTTP proxy between infrastructure and Datadog, enabling users to launch tests as part of their CI/CD pipelines. The tunnel is designed to support CI pipelines and local development, allowing teams to verify hotfixes or new features locally before committing code or running tests in environments reserved for CI pipelines. Private locations are Docker containers that can be deployed as custom points of presence inside an organization's infrastructure using orchestration tools like Docker Compose, Kubernetes, AWS Fargate, and Amazon ECS. They provide a durable probing service for launching tests and can be useful for customizing and managing a centralized testing tool across the organization, triggering tests on long-running environments as part of CI/CD pipelines, and regularly running tests on internal applications hosted on private networks to ensure availability SLOs are maintained. Both private locations and the testing tunnel offer unique features to support different testing goals, with private locations being more suitable for long-term testing and monitoring, while the testing tunnel is ideal for rapid, on-demand testing in short-lived environments.

Datadog has introduced cross-browser testing support for Firefox, Chrome, and Microsoft Edge in its Synthetic Monitoring tool, with Internet Explorer 11 available in private beta. This feature allows users to create code-free end-to-end browser tests by recording user journeys and adding assertions. The multi-locator targeting algorithm ensures that the tests do not fail due to frontend changes. Cross-browser tests can be run automatically on a range of devices and in any environment, including CI pipelines, enabling teams to maintain consistent experiences for users across different browsers.

Datadog CI Visibility provides critical visibility into your organization’s CI/CD workflows, enabling you to monitor pipeline builds, stages, and jobs, track test performance, identify flaky tests, analyze test performance, and ensure smooth, reliable builds. With Datadog's turn-key CI provider integrations and synthetic tests in CI pipelines, it complements existing tools and gives deep insight into key pipeline metrics. This solution helps organizations understand the performance of their CI pipelines, making it easy to identify issues and make workflows faster and more reliable.

Datadog's testing tunnel and private locations offer flexible solutions for internal application testing, enabling organizations to create custom probes with private locations for durable testing and monitoring, and leveraging the testing tunnel for on-demand testing in local and continuous integration environments. The testing tunnel is an NPM package that leverages Datadog's command line interface to create an end-to-end encrypted HTTP proxy between infrastructure and Datadog, allowing organizations to launch tests as part of their CI/CD pipelines and identify and fix regressions before they impact users. Private locations are durable probing services deployed as custom points of presence inside of infrastructure using orchestration tools like Docker Compose, Kubernetes, AWS Fargate, and Amazon ECS, providing a centralized testing tool for teams across the organization and enabling organizations to trigger tests on long-running environments such as staging and pre-production.

Datadog's Live Processes feature provides insight into workloads by tracking resource consumption metrics, traces, and network data for each process running in an infrastructure. It now correlates multiple data types by PID to help identify issues such as network bandwidth saturation, application errors, and infrastructure latency. The Live Processes view allows users to investigate resource constraints on applications, monitor memory usage, and pinpoint the root cause of network issues. By integrating with Network Performance Monitoring, APM, Log Management, and more, Datadog offers comprehensive visibility into process-level activity without additional configuration.

Datadog's Live Processes provides insight into workloads by tracking resource consumption metrics, traces, and network data for each process running in the infrastructure. It correlates multiple data types by PID to identify issues such as network bandwidth saturation, application errors, infrastructure latency, and other problems in the system. This feature helps teams quickly find the scope of an issue, notify relevant teams, and investigate next steps before end-users are affected. By correlating processes with distributed tracing and APM data, Datadog enables users to easily determine which applications are facing resource constraints or using more resources than expected. The Live Processes view also allows users to inspect traces generated by a process, view logs for applications on problematic hosts, and monitor the performance of related processes in real-time. This comprehensive visibility into process-level activity helps teams respond quickly to issues and prevent similar problems in the future.

Multi-factor authentication (MFA) is an increasingly popular method for securing user accounts, requiring users to provide two or more pieces of identifying information when logging into an application. This can include unique verification links or codes sent to the user's phone or email address, as well as time-based one-time passwords (TOTPs). However, testing features with MFA enabled can be challenging. Datadog now supports automatic generation and use of TOTPs in synthetic tests, allowing users to fully test their application's MFA modules and features without disabling critical security measures or manually entering authentication codes created by separate tools. This also eliminates the need for creating and maintaining dedicated environments to test MFA-enabled user journeys. Additionally, Datadog enables testing of other authentication methods such as sending one-time codes via phone or email, and generating custom synthetic email addresses for simple validation workflows. Detailed information about every test run is provided, allowing users to quickly identify bugs in their authentication workflows.

Datadog has introduced the ability to automatically generate and use time-based one-time passwords (TOTPs) as a form of multi-factor authentication directly in Synthetic browser and API tests. This feature complements existing support for testing other authentication methods, eliminates the need for creating and maintaining dedicated environments to test MFA-enabled user journeys, and enables full testing of application's MFA modules and features without disabling critical security measures or manually entering authentication codes created by separate tools. Additionally, Datadog can generate custom synthetic email addresses for simple validation workflows, such as sending a verification link to a user's inbox after they sign up for a new account, and provides detailed information about test runs to quickly identify bugs in authentication workflows before they affect end users.

President Biden signed an executive order on May 12, 2021, emphasizing the importance of improving federal agencies' cybersecurity practices following recent high-profile attacks. The order includes several key points such as prioritizing cybersecurity, enhancing supply chain security, and encouraging cloud adoption with adherence to Zero Trust architecture. It also promotes sharing threat information between public and private sectors, standardizing incident response playbooks, and improving detection capabilities through endpoint detection and response initiatives.

The executive order signed by President Biden on May 12, 2021, emphasizes the importance of improving federal agencies' cybersecurity practices to prevent severe economic and civilian impacts. The order calls for close partnership between the public and private sectors to create a more secure cyberspace, with a focus on supply-chain security, Zero-Trust architecture, and cloud adoption. Key areas of emphasis include removing barriers to sharing threat information, modernizing federal government cybersecurity, enhancing software supply chain security, establishing a Cyber Safety Review Board, standardizing vulnerability and incident response playbooks, improving detection of cybersecurity vulnerabilities and incidents, and strengthening investigative and remediation capabilities. The order also requires agencies to adopt multi-factor authentication, encryption, and secure cloud services, such as infrastructure-as-a-service, platform-as-a-service, and software-as-a-service.

The Request Flow Map is a new feature by Datadog that provides a live view of requests between services across an entire environment. It supports filtering by any tag and allows users to pivot seamlessly to service-level details, enabling quick identification of the root cause of performance issues. The map leverages application traces to construct a real-time view of service dependencies, making it easier to navigate complex environments. Users can also click on services for more granular details about their performance and pivot between monitors, runtime metrics, traces, and synthetic tests for deep context when troubleshooting.

Datadog APM introduces the Request Flow Map, a feature that provides a live view of requests between services across an entire environment. The map allows users to search and analyze traces in real-time, filter by tags, and pivot to service-level details to identify root causes of performance issues quickly. With the Request Flow Map, users can follow errors to their source, gain insights into system architecture, and troubleshoot complex distributed systems more efficiently.

The 2021 RSA Conference, held virtually for the first time, highlighted the growing marketplace demand for integrating operational visibility and security, particularly through the roles of chief product security officers and DevSecOps, despite lacking a universally accepted definition. The conference emphasized the challenges of unifying security and DevOps, focusing on shared visibility, tooling, and understanding, while addressing the persistent divide between their mindsets. Discussions underscored the importance of managing attack surfaces and highlighted the high incidence of cloud breaches due to misconfigurations. The concept of defense in depth was reinforced as essential, advocating for layered security measures to protect against infiltration and post-infiltration threats. The Datadog Cloud Security Platform was presented as a solution to enhance resilience by offering full-stack security, promoting collaboration between DevOps and security teams, and providing tools like Cloud Security Posture Management and Cloud Workload Security for continuous monitoring and threat detection.

The DevSecOps Maturity Model is a structured framework that helps organizations evaluate their current level of DevSecOps maturity and identify steps to improve it. It covers more than 30 key capabilities across two major areas: People & Culture, and the phases of the end-to-end DevSecOps lifecycle. A 10-minute online self-assessment is available for teams to gauge their current level of maturity and set goals for the future. The model aims to help organizations adopt agile workflows and integrate security into the development and delivery process, breaking down silos between development, operations, and security teams.

Datadog has developed a DevSecOps Maturity Model to help organizations assess their current level of maturity and identify steps to improve it. The model, based on the company's collective experience working with thousands of customers, takes a holistic approach that covers key practices across development, security, and operations teams. It provides a structured framework to answer three key questions: what is the current maturity, where do they want to be, and how to bridge the gap. A 36-question self-assessment tool is available online for quick diagnostic results, along with recommendations for advancing to the next level of maturity. The model also includes four major levels of maturity for each capability, such as patching systems, and provides a white paper with granular detail on each competency area. Datadog's unified platform can help teams break down silos while securing their environment, and a 14-day free trial is available for non-customers.

ASP.NET Core is an open source web development framework that enables developers to create .NET applications on various platforms such as macOS, Linux, and Windows machines. The introduction of .NET Core in 2016 has increased the number of ways to build and deploy .NET applications. Monitoring application performance across a wide variety of platforms is crucial for identifying issues with services and their dependencies. Datadog's .NET tracer uses the .NET profiling APIs to add out-of-the-box instrumentation for many common libraries and programming languages used for both .NET Core and .NET frameworks, including VB.NET, C#, and F#. This guide demonstrates how to instrument a sample containerized ASP.NET Core application to send traces to Datadog for monitoring by deploying the Datadog Agent in a Docker container and using Datadog's .NET tracer.

In this post, the authors walk through how to instrument a sample containerized ASP.NET Core application to send traces to Datadog for monitoring. They cover deploying the Datadog Agent in a Docker container and configuring it to collect application performance data across various platforms like Linux and Windows machines. The authors also show how to use Datadog's .NET tracer, which provides out-of-the-box instrumentation for many common libraries and programming languages used for ASP.NET Core and .NET frameworks. They provide step-by-step instructions on how to set up the .NET tracer in both Linux and Windows containers, including creating a Dockerfile and updating the docker-compose.yml file. The authors demonstrate how to run the instrumented application and view its performance data in Datadog APM, highlighting features like distributed traces, key metrics visualization, and integration with .NET logs. Overall, this post provides a comprehensive guide on how to monitor ASP.NET Core applications using Datadog's containerized Agent and .NET tracer.

Vercel is a platform that enables developers to build high-performance hybrid websites and applications, offering built-in deployment tools, an Edge Network for fast retrieval, and Serverless Functions for backend processes. In partnership with Datadog, users can now monitor their Vercel Serverless Functions' performance and analyze function logs alongside logs from across their infrastructure using the Log Explorer. Custom function logs can be used to generate insights into business operations. Additionally, Datadog Synthetic tests can be integrated into deployment pipelines to monitor frontend performance of applications powered by Vercel. This comprehensive solution provides full visibility into the health and performance of Vercel-powered applications.

Vercel is a deployment and collaboration platform that enables frontend developers to build high-performance hybrid websites and applications. Vercel also offers Serverless Functions, which allow users to deploy serverless code to accomplish essential backend processes like user authentication, form submission, and database queries. The platform's Edge Network caches sites for fast retrieval. Datadog has partnered with Vercel to offer a comprehensive monitoring solution that enables users to ingest function logs for analysis and long-term storage, as well as create browser tests to track frontend performance. Users can monitor their function performance using request logs, visualize key application data by emitting custom function logs, and monitor their Vercel-powered frontend with browser tests. Additionally, Datadog allows users to easily ingest Vercel logs for full visibility into the health and performance of their Vercel Serverless Functions.