November 2022 Summaries

The text discusses the challenges faced by security organizations in securing their cloud environments, particularly with regards to alert fatigue and the increasing complexity of large cloud and multi-cloud environments. To address this issue, Datadog is announcing the Essential Cloud Security Controls Ruleset for Cloud Security Management (CSM), which provides a set of 10 key rules each for AWS, Azure, and GCP to help security teams prioritize remediation of misconfigurations in their environment. The rules are based on industry best practices, risk of significant impact, and potential to have prevented known public breaches. By using this ruleset, DevOps and security teams can streamline their security efforts, reduce alert fatigue, and minimize the risk of high-criticality misconfigurations slipping through the cracks. Additionally, Datadog's Security Research team continuously evaluates the threat landscape for new best practice configurations, cloud breaches, and cloud attack toolsets used by security researchers and threat actors, ensuring that customers stay up-to-date and protected from emerging risks.

Amazon Security Lake enables customers to create security data lakes from various sources and integrate them with third-party SIEM solutions like Datadog Cloud SIEM. This integration allows users to analyze security logs in real-time for threat detection, using out-of-the-box detection rules, dashboards, and log management tools. The setup process is quick and easy, enabling users to start analyzing their security logs within minutes. With this integration, Datadog can collect and visualize logs from Security Lake, helping users identify and investigate threats across their environment. Additionally, saved views in Log Explorer allow users to focus on specific subsets of logs for faster troubleshooting.

Datadog has achieved TISAX Assessment Level 2 (AL2) certification, confirming its ability to handle data with high protection needs according to the strict TISAX standards maintained by the ENX Association. This enables organizations in the automotive industry and its supply chain to confidently rely on Datadog as a comprehensive, TISAX-compliant monitoring solution. The Trusted Information Security Assessment Exchange (TISAX) is an evaluative framework that defines and implements information security standards for the automotive industry and its entire supply chain. With this certification, Datadog continues to provide secure, compliant, and comprehensive monitoring solutions for various industries.

Cloud solutions offer flexibility but managing resource utilization can be challenging, leading to over or under provisioning and increased costs. Datadog's integration with AWS Compute Optimizer provides memory utilization metrics that enhance EC2 instance recommendations for optimal performance at an optimal price. By leveraging this integration, users can streamline their cloud spend while improving application performance. Additionally, Datadog APM helps monitor the impact of provisioning changes on applications and alerts to any issues.

Embrace is a mobile application monitoring solution that combines data analytics, real user monitoring, network performance monitoring, and hardware monitoring in one platform. The Embrace Datadog app allows users to track and troubleshoot mobile app performance by collecting KPIs for crashes and HTTP errors from their mobile applications alongside key debugging context such as stack traces and the availability of specific endpoints. This integration enables users to spot ongoing regressions and quickly initiate root cause analysis. The Embrace Datadog app is seamlessly integrated with Datadog's full-stack observability platform, allowing users to view this data side-by-side with network telemetry and performance metrics from the backend infrastructure that their mobile apps rely on.

Universal Service Monitoring (USM) is a tool that automatically detects all services running on an infrastructure and monitors their golden signals to provide visibility into service health, performance, and dependencies without requiring any code changes. USM helps organizations track service dependencies using the Service Map, centralize knowledge of services with the Service Catalog, create alerts and SLOs for faster incident response, and troubleshoot issues more efficiently by leveraging logs, APM, and Database Monitoring. By providing RED metrics for all services, USM enables proactive monitoring and automatic cataloging of new and existing services to prevent observability gaps.

Datadog has partnered with AWS Compute Optimizer to create an integration that improves Amazon Elastic Cloud Compute (EC2) instance recommendations. This integration allows users to get memory-aware EC2 instance recommendations, which helps ensure the best possible performance at an optimal price. By surfacing cloud cost trends alongside performance metrics and usage patterns, Datadog gives users context to track performance across their cloud infrastructure and streamline spend. With this integration, users can unlock additional instance types in their recommendations, giving them insights tailored to their actual usage. Additionally, Datadog APM enables users to monitor provisioning impacts and identify issues caused by incorrectly provisioned resources. The partnership between Datadog and AWS Compute Optimizer helps users get higher quality EC2 instance recommendations with memory utilization data from the Datadog agent, streamlining cloud cost while improving application performance.

Embrace is a mobile app monitoring solution that combines data analytics, real user monitoring, network performance monitoring, and hardware monitoring to provide a single platform for tracking and troubleshooting mobile app performance. The Embrace Datadog app offers an out-of-the-box integration with Datadog's full-stack observability platform, enabling users to spot ongoing regressions and quickly kick off root cause analysis. With the app, teams can track and investigate the root cause of crashes, view network telemetry from their apps side-by-side with backend network metrics, and understand their apps' network performance. By using Embrace with Datadog, mobile engineers and DevOps staff can get full-stack observability into the health and performance of their mobile apps, tracking crashes, networking issues, and more from a unified slate of visualizations and monitoring tools.

Amazon Security Lake allows customers to build security data lakes from various sources, including cloud and on-premises data. This integration enables customers to forward security logs from Security Lake to Datadog Cloud SIEM for real-time threat detection and analysis. The setup is quick and easy, requiring only a few minutes to install the AWS integration and update IAM role permissions. With this integration, customers can leverage out-of-the-box (OOTB) security detection rules, dashboards, and Log Management tools to identify and investigate threats across their environment. Datadog Cloud SIEM provides signals associated with suspicious activity, which are mapped against IP addresses and attack patterns, enabling swift analysis of security-related telemetry data from Security Lake. Additionally, customers can use saved views to focus on specific subsets of logs and quickly highlight security-related issues for faster troubleshooting.

Datadog has achieved TISAX Assessment Level 2 (AL2) certification, confirming its ability to handle sensitive data with high protection according to strict automotive industry standards. This certification enables organizations in Germany and elsewhere to rely on Datadog as a comprehensive, TISAX-compliant monitoring solution for their service and application infrastructure. The Trusted Information Security Assessment Exchange (TISAX) is an evaluative framework that defines information security standards for the automotive industry, specifying data protection and exchanges with third parties, which are critical when handling sensitive customer data. Datadog's certification demonstrates its commitment to providing a secure, compliant, and comprehensive monitoring solution for customers, including those in the healthcare and life science industries. With this achievement, organizations can now trust Datadog to uphold strict TISAX security standards while providing end-to-end visibility into their cloud-based infrastructure, applications, and services.

AWS Lambda SnapStart is a new feature that improves startup performance for latency-sensitive Java applications by up to 10 times at no extra cost, and typically without modification to function code. Datadog Serverless Monitoring now supports AWS Lambda SnapStart, allowing teams to efficiently mitigate cold starts by enabling SnapStart for their functions and using Datadog Serverless Monitoring to monitor how the new AWS Lambda feature affects application performance. This partnership provides enhanced visibility into serverless applications' performance, including the rate of cold starts, and enables users to create alerts based on specific performance indicators.

Datadog has introduced Cloud SIEM Investigator for AWS environments, with support for other major cloud providers coming soon. The tool leverages AWS CloudTrail logs to help teams visualize activity associated with AWS entities such as IAM users, roles, resources, and more. It provides a centralized view of the who, what, when, and how behind changes in cloud environments, enabling DevOps and security teams to investigate issues effectively. The Investigator is integrated with both Log Explorer and Security Signals, allowing disparate teams to work together on identifying the source of flagged events or logs. This shared context improves collaboration on investigations and helps identify the root cause of changes faster.

AWS Lambda enables engineering teams to build modern and scalable services without provisioning underlying infrastructure resources. Monitoring Lambda functions requires visibility into performance indicators that differ from traditional architectures, particularly cold starts which occur when a serverless compute platform like Lambda needs to create a new execution environment to serve a request. Datadog Serverless Monitoring detects cold starts in Lambda functions, visualizes their impact on services via distributed tracing, and lets users set alerts based on the rates at which cold starts occur. The partnership between AWS and Datadog enables support for AWS Lambda SnapStart, a feature that improves startup performance for latency-sensitive Java applications by up to 10 times without modification or extra cost. Teams can use Datadog Serverless Monitoring to detect cold starts, reduce cold start latency with Lamba SnapStart, and gain complete visibility into the impact of cold starts on Lambda functions.

Datadog has announced Datadog Cloud SIEM Investigator for AWS environments, which helps organizations streamline their investigations of activity in cloud-native infrastructure. The tool leverages AWS CloudTrail logs to provide a centralized view of activity associated with AWS entities, enabling DevOps and security teams to visualize the who, what, when, and how behind changes in their cloud environments. This visibility provides shared context for teams to improve collaboration on investigations, effectively identifying the root cause of changes faster, while also reducing false positive alerts on sensitive resources.

Datadog's Error Tracking feature for logs, now available within Datadog Log Management, enhances the ability to identify and prioritize software bugs by intelligently grouping related errors into issues. This tool integrates with Real User Monitoring (RUM) and Application Performance Monitoring (APM), providing diagnostic data like stack traces and error distributions to help pinpoint root causes. Users can manage and sort issues based on error occurrences or age, and utilize workflow states to track remediation progress. Error Tracking allows for deeper dives into individual issues, offering context such as historical error volumes and source code insights. Additionally, it enables the creation of monitors for detecting new issues or high error counts, with alerts sent via integrations like Slack and PagerDuty. This functionality helps reduce noise, accelerate root cause identification, and decrease resolution times, ultimately improving software reliability.

Cloudcraft is a real-time cloud infrastructure modeling solution that enables users to visualize their cloud infrastructures and create dynamic architecture models. It provides insights into configurations, cost, and inter-service interactions. The tool automatically updates diagrams when the infrastructure changes. Datadog has recently acquired Cloudcraft, aiming to integrate its real-time observability data with Cloudcraft's capabilities to provide robust, dynamic architecture diagrams for customers. This integration will support cloud migrations, container adoptions, and other architectural changes.

Cansu Berkem announces that Cloudcraft, a powerful real-time cloud infrastructure modeling solution, is joining Datadog to provide robust and dynamic architecture diagrams. Cloudcraft offers real-time visualization of cloud infrastructures, enabling users to build dynamic architecture models that detail configurations, cost, and inter-service interactions. The combination of Datadog's real-time observability data with Cloudcraft will allow customers to shift monitoring even further left, supporting cloud migrations, container adoptions, and other common architectural changes.

Part 2 of a cloud security series by Mallory Mooney emphasizes the critical task of securing endpoints within an organization's network, especially in cloud environments where the complexity and variety of endpoints have significantly increased. It highlights the importance of mapping all connected endpoints, implementing effective management practices, and gaining visibility into endpoint activity to prevent active threats. Endpoint vulnerabilities often arise from outdated software, weak passwords, or misconfigurations, which threat actors can exploit through methods like phishing or active scanning. To mitigate these threats, organizations are advised to adopt the Zero Trust model to control access based on identity and least privilege principles, and to use CIS benchmarks for fine-tuning endpoint configurations. The text also emphasizes the significance of integrating security monitoring solutions like Endpoint Protection Platforms, SIEM, CWS, and CSPM to enhance visibility and threat detection across all endpoints, ensuring comprehensive endpoint security in cloud-native infrastructures.

React Native has become a popular development framework for cross-platform mobile applications due to its ability to interact with native APIs and require minimal platform-specific code. However, detecting and troubleshooting errors in React Native applications can be complex as they may originate from either JavaScript or native iOS/Android code. Datadog's Crash Reporting and Error Tracking are now available for React Native via Real User Monitoring (RUM), providing comprehensive human-readable error reports, crash detection, analytics, and alerts on error trends. This feature generates source maps for your code automatically upon each build of the application, symbolizing stack traces and unminifying code to create clear error reports. RUM also provides detailed visibility into the context in which errors occur within user sessions. Enabling Crash Reporting ensures comprehensive oversight of all potential error sources for React Native applications, helping developers quickly get to the root of errors and improve application stability.

The text explains how to enhance SQL Server monitoring using Datadog by collecting custom metrics, which allows for a more tailored approach to monitoring and improving database performance. It outlines how to configure the Datadog Agent to collect additional performance counters beyond the default ones, using both the SQL Server integration and Windows Management Instrumentation (WMI) integration. By editing configuration files, users can specify custom metrics and associate them with specific performance counters or WMI classes, allowing for granular tracking of database activities such as page lookups, log flushes, and failed SQL Server jobs. The article highlights the value of combining these custom metrics with Datadog's broader monitoring capabilities to identify performance issues and optimize databases effectively. It also mentions that users can try Datadog's services with a 14-day free trial to monitor SQL Server and over 650 other technologies.

React Native has become the predominant development framework for cross-platform mobile applications, allowing developers to build applications for iOS, Android, and the browser using the same declarative JavaScript. However, this cross-platform adaptability also presents challenges, such as detecting and troubleshooting errors that can be difficult due to minified or unsymbolicated code. To address these issues, Datadog Crash Reporting and Error Tracking are now available for React Native via Real User Monitoring (RUM), providing comprehensive human-readable error reports, crash detection, analytics, and alerts on error trends. With this integration, developers can easily identify recurring errors, pinpoint bugs in their JavaScript, native iOS, and native Android code, and receive critical error alerts, allowing them to simplify and expedite critical troubleshooting and gain continuous insight into their React Native applications.

Datadog Real User Monitoring (RUM) has introduced the ability to generate metrics from all RUM events, allowing users to visualize trends in user activity and application health over a period of 15 months. This feature enables users to analyze historical data for seasonal traffic patterns and year-to-year growth, track service level objectives (SLOs), and configure alerts for anomalous behavior. By leveraging RUM-based metrics, users can gain full insight into their applications' performance and user experience.

Datadog Real User Monitoring (RUM) allows users to capture and retain complete user sessions for 30 days, enabling the identification of bugs, prioritization of issues, and determination of fixes. With RUM-based metrics, users can visualize 15-month trends in user activity and application health, helping them determine the health of their application throughout the year. Users can generate custom metrics from all RUM events, allowing them to analyze historical trends, track service level objectives (SLOs), and be alerted to anomalous user and application behavior. These metrics are retained for 15 months, enabling users to compare seasonal traffic patterns and year-to-year growth. By using RUM-based metrics, users can establish SLOs to maintain a high-quality user experience, visualize their user experience objectives alongside other essential application metrics, and configure alerts for anomalous behavior.

The text discusses Datadog's Log Explorer and its new feature, Pattern Inspector, which enhances the process of analyzing log data by allowing users to view the distribution of actual values hidden within log patterns. This functionality is particularly useful for troubleshooting incidents and performing security audits, as it simplifies the identification of patterns and anomalies in large datasets. By hovering over highlighted sections in log patterns, users can quickly discern key information such as the distribution of IP addresses or error codes, which can aid in pinpointing the root cause of issues or identifying potential security threats. Pattern Inspector significantly expedites the investigative process by enabling users to isolate and address problems more efficiently, thereby improving the overall user experience and reducing the time taken to resolve issues.

Speedscale is a testing framework for Kubernetes applications that enables load testing with real-world production scenarios by replaying actual API traffic. It allows customers to capture snapshots of real-time traffic and use them to generate sanitized load tests without any scripting required. The software license for Speedscale can now be purchased in the Datadog Marketplace, allowing users to view their Speedscale reports alongside the rest of their application telemetry hosted in Datadog APM. This integration enables efficient troubleshooting of poor performance and visualization of key traffic replay data using an out-of-the-box dashboard.

Datadog has introduced Error Tracking to help developers make sense of errors and prioritize troubleshooting efforts. The feature automatically groups application errors into issues, allowing users to investigate more effectively and reduce the mean time to resolution. Error Tracking can be used with frontend JavaScript as well as Android applications and all languages supported by Datadog's distributed tracing and APM. It extracts error messages from RUM and APM data without requiring any additional SDK configuration or application code modification. The Error Tracking Explorer view provides a list of detected issues, along with important aggregates such as total error count and frequency over time. By grouping errors into issues and showing where they arise in the application source code, Error Tracking can help developers identify trends that may otherwise go unnoticed.

Speedscale is a testing framework built for Kubernetes applications that enables load testing with real-world production scenarios by replaying actual API traffic. It allows customers to capture snapshots of real-time traffic and generate sanitized load tests without scripting required. The framework's integration with Datadog APM enables users to view Speedscale reports alongside their application telemetry, facilitating efficient troubleshooting. Speedscale's traffic replay feature simulates incoming requests to service endpoints, allowing for testing of varied conditions such as high throughput, faults, and response latency. By ingesting Speedscale reports as Datadog Events, users can track report history and correlate performance against historical application metrics in Datadog APM. The integration also enables the visualization of key traffic replay data using an out-of-the-box dashboard, providing a high-level overview of recent traffic replays and enabling quick monitoring of service performance.

In the Datadog Spotlight, Keegan Cotton, an Enterprise Account Executive for Major Accounts based in Houston, Texas, shares insights about his career journey and role at Datadog. With a background that includes serving in the US Army and working in various international roles, Keegan transitioned into the tech industry after earning an MBA and engaging in ventures like a renewable energy startup. At Datadog, he emphasizes the importance of understanding customer needs and balancing work with personal life. Keegan highlights transferable skills like passion, perseverance, and teamwork, particularly valuable for veterans transitioning to tech careers. He is actively involved with Datadog's Veterans Community Guild, which focuses on veteran support and engagement. Keegan encourages veterans and other candidates interested in joining Datadog to connect with the company’s supportive community, noting the organization's commitment to attracting diverse talent.

In 2021, Datadog partnered with AWS to develop the Lambda extension, which simplifies and reduces costs for collecting traces, logs, custom metrics, and enhanced metrics from Lambda functions and submitting them to Datadog. With the release of the Lambda Telemetry API, the latest version of the Datadog Lambda extension now provides deeper visibility into the performance of your Lambda functions by visualizing the impact of cold starts with cold start trace spans and offering new enhanced metrics such as ResponseDuration, ResponseLatency, and ProducedBytes. These additional features complement existing monitoring capabilities and help users spot and troubleshoot Lambda function performance issues more effectively.

The AWS Lambda Telemetry API has been released, expanding the capabilities of the Datadog Lambda extension to provide deeper visibility into the performance of Lambda functions. With this update, users can now visualize the impact of cold starts on their functions with cold start trace spans and gain even deeper insights into Lambda function performance with new enhanced metrics. These additional features enable users to monitor and troubleshoot Lambda function issues more effectively, spotting potential problems before they become critical. The Datadog Lambda extension now provides an expanded look into Lambda function performance telemetry, allowing users to monitor this data alongside CloudWatch Lambda metrics and logs from other AWS services via the Datadog Forwarder function.

Guy Arbitman's article discusses the complexities and challenges of monitoring HTTP sessions using traditional network capture tools like tcpdump, which are often inefficient and resource-intensive. The article introduces the Extended Berkeley Packet Filter (eBPF) technology, which provides a more effective solution by allowing traffic capture directly from the Linux kernel, thus offering significant improvements in monitoring HTTP sessions with minimal performance impact. eBPF's ability to run in the kernel space enables more complex processing of application-layer traffic, overcoming the limitations of classic Berkeley Packet Filter (BPF), which is restricted to user space and less efficient for handling HTTP sessions. Additionally, the article provides a walkthrough for building an eBPF-based protocol tracer, using a REST API server example written in Go, to efficiently capture and analyze network traffic. It highlights eBPF's broader applicability, as seen in its use by Datadog for enhanced visibility into application-level traffic without code instrumentation, and its acquisition of Seekret to further leverage eBPF for API management across environments.

Google Kubernetes Engine (GKE) is a managed Kubernetes service that allows users to deploy and orchestrate containerized applications on Google's infrastructure. Datadog has introduced new features for its GKE integration, enabling users to monitor all components of their clusters alongside control plane metrics directly through the GKE integration. The integration comes with two out-of-the-box dashboards (Standard and Enhanced) that visualize key cluster metrics at various levels, including nodes, pods, containers, and applications. Additionally, Datadog's Kubernetes integration provides built-in support for Kubernetes State Metrics, allowing users to see detailed information about the status of their pods, nodes, and deployments without running a kube-state-metrics service within their cluster. The Agent also ingests Kubernetes logs and events, which can be correlated with distributed traces and service-level metrics from containerized applications. By monitoring control plane components and key metrics, users can diagnose and debug issues in their GKE clusters more effectively.

Datadog's GKE integration provides deep visibility into the health and performance of Kubernetes clusters, enabling users to monitor and alert on important metrics such as CPU, memory, volume, and ephemeral storage usage, as well as cluster health metrics. The integration comes with two out-of-the-box dashboards that visualize key cluster metrics, including control plane components, which provide insight into platform issues and help diagnose and debug issues in GKE clusters. Users can also ingest additional metrics through the Datadog Agent, providing even deeper visibility into Kubernetes resources and workloads. The integration is bundled together with the GCP integration and can be easily installed to start monitoring new GKE metrics today.

This article discusses the monitoring of MongoDB performance with the WiredTiger storage engine. It provides an overview of NoSQL databases and explains how MongoDB works as a document-oriented database. The key areas to track and analyze metrics are outlined, including throughput metrics, database performance, replication and oplog, journaling, concurrent operations management, cursors, resource utilization, storage metrics, memory metrics, other host-level metrics, and resource saturation. The article also covers scaling MongoDB using sharding versus replication.

OAuth is now fully supported by Datadog API-based data integrations, allowing third-party applications to securely request access to users' Datadog data and submit data on their behalf without requiring sensitive credentials like API or application keys. This makes it easier for Datadog users to enable data integrations and seamlessly transfer data between Datadog and other tools. Three successful examples of OAuth usage in Datadog integrations include Vantage, LambdaTest, and Adaptive Shield. These partnerships leverage OAuth to ensure secure and seamless data transfers while reducing the risk of sensitive data leaks and simplifying user experience.

Datadog has now fully implemented OAuth support for its API-based data integrations, allowing third-party applications to securely request access to users' Datadog data without requiring sensitive credentials. This enhancement enables seamless and secure data transfer between Datadog and third-party applications, making it easier for customers to enable data integrations and start seeing data flow back and forth. Three key partners, Vantage, LambdaTest, and Adaptive Shield, are successfully leveraging OAuth to build integrations that provide secure, seamless, and intuitive user experiences. With OAuth, these partners can request specific scopes of data rather than relying on individual users to generate API keys, reducing vulnerability and eliminating potential for user-error. This industry-standard security protocol offers a quick and painless process for getting integrations up and running, instilling confidence in customers and streamlining the end-user experience.