April 2021 Summaries

Datadog developed an automated auditing tool called "Clarity" to manage and monitor the security and efficiency of SaaS applications used by employees. Clarity audits user accounts in various SaaS applications against Datadog's HR system, Workday, to identify any discrepancies or unauthorized access. The tool integrates with AWS Lambda, Datadog, Slack, and Freshservice to log activities, send notifications, and create tickets for flagged accounts, ensuring quick remediation. By utilizing Datadog's metrics and logs, Clarity provides detailed insights and allows for seamless integration into existing workflows, ultimately enhancing visibility and control over the company's SaaS portfolio. Future plans include expanding Clarity's capabilities to monitor the actual usage of accounts, aiming to optimize software license expenditure and access management.

Datadog has partnered with AWS to launch their managed Lambda Layer for OpenTelemetry, extending OpenTelemetry tracing to Lambda functions. This collaboration allows users to collect OpenTelemetry traces from Python, Node.js, and Java Lambda functions without any application code changes. The Lambda Extensions feature offers a seamless way for monitoring and security tools to integrate with the Lambda execution environment. By using this managed Lambda Layer, developers can focus on building their applications while maintaining high performance and minimizing operational complexity.

Datadog's Network Performance Monitoring (NPM) feature automatically detects and tags AWS- and GCP-managed endpoints in your network, providing visibility into cloud service dependencies. With this feature, you can visualize your cloud architecture, monitor cloud service health, assess the effect of poor managed database performance on app health, distinguish between client-side and provider-side issues, pinpoint when S3 and RDS errors affect application health, and monitor cloud dependencies from every angle. This enables you to quickly identify the root cause of poor application performance and take corrective action to improve your application's reliability and performance.

Datadog Real User Monitoring (RUM) is a tool that provides detailed insight into user behavior and application performance, regardless of whether applications run on mobile devices or in the browser. It supports React Native monitoring, allowing developers to track errors, crashes, launch time, user actions, and network requests, as well as get comprehensive visibility into user behavior, determine which errors need investigation first, and find causes of long load times and dropped frames. With Mobile RUM, users can investigate errors faster, identify performance bottlenecks, and discover the sources of slow refresh rates for both native and JavaScript processes. The tool also provides unified visibility into React-based applications, whether they run in the browser or on iOS and Android devices.

Detecting unauthorized access to an AWS account is crucial for maintaining security. One common method of gaining unauthorized access is through the use of assumed roles, which can be difficult to track due to the permission model in AWS environments. Datadog Cloud SIEM offers a solution by automatically detecting when a user assumes a role, allowing users to determine whether the role change is legitimate or not. This helps prevent potential security threats from escalating further. By using term detection methods and setting up rules that monitor for unfamiliar accounts assuming roles, users can receive alerts and investigate any suspicious activity in their cloud environments.

In the context of AWS security, detecting unauthorized access to an account is crucial. This can occur when a third-party tool is granted access to monitor infrastructure or optimize bills, making it hard to track due to permission models. Datadog Cloud SIEM offers a solution by automatically detecting when a user assumes a role, allowing teams to investigate and take action before the threat propagates further. The platform analyzes log data over a chosen period to establish a baseline of expected behavior, generating Security Signals for anomalous activity. By setting up a term-based rule, teams can be alerted whenever an unfamiliar AWS account assumes a role in their environment, enabling swift investigation and response.

HashiCorp Vault is a tool that helps manage, store, and secure access to secrets such as encryption keys, credentials, certificates, and tokens. It has been adopted by organizations like Adobe, Hulu, and Shopify since its release in 2015. Vault enables security operators to encrypt all of their secrets, distribute them across hybrid environments, apply fine-grained access controls, and audit activity to see who has requested data. Built to operate in zero-trust networks, Vault takes an application identity-centric approach, meaning that it authenticates clients against trusted sources of identity (e.g., Google Cloud, GitHub, Okta) before granting them access to data. Teams can deploy Vault on platforms like Kubernetes and AWS to streamline secret management operations across their entire stack or take advantage of HashiCorp Cloud Platform’s fully managed products. Monitoring the health and performance of your Vault deployment is crucial for protecting applications and infrastructure against potential attacks. In this blog post, we discuss how you can gain visibility into the health and performance of your Vault deployment with metrics and logs. We also explore how to use built-in Vault monitoring tools to view these metrics in detail. The general architecture of Vault consists of several components, including its core, storage backend, audit devices, and replication mechanisms. Monitoring various parts of Vault's architecture is essential for keeping your Vault clusters healthy and running optimally. In this post, we cover the following topics: 1. Overview of Vault logs: We discuss two types of logs produced by Vault - server logs and audit logs. Server logs record all activities that occurred on each server, while audit logs record the requests and responses of every interaction with Vault. 2. Built-in tools for collecting and viewing Vault metrics and logs: In Part 2 of this series, we introduce the built-in tools you can use to collect and view Vault metrics and logs. We also provide examples of how these tools work in practice. 3. Comprehensive visibility into Vault using Datadog: In Part 3, we show you how to use Datadog to analyze your logs - and seamlessly correlate them with metrics and other data from across your stack to get comprehensive visibility into your applications. We also demonstrate how Datadog Cloud SIEM automatically analyzes Vault audit logs to ensure that your Vault cluster is secure. By monitoring various parts of Vault's architecture, you can ensure the health and performance of your Vault deployment, protect your applications and infrastructure against potential attacks, and gain comprehensive visibility into your entire stack.

HashiCorp Vault is a complex system composed of multiple components, and monitoring its health and performance requires tracking these parts along with the resources they consume. Datadog offers a comprehensive solution for monitoring Vault, featuring over 650 integrations, advanced alerting, and log analytics. By using Datadog, users can automatically detect potential security threats, correlate data across their stack, and track long-term performance trends. The platform allows for the collection and retention of metrics and logs from Vault, providing deep insights into the cluster's activity and health. Datadog's integration with Vault facilitates setting up customizable dashboards and automated alerts, enabling users to effectively monitor Vault's performance and availability, as well as detect and troubleshoot issues quickly. Additionally, Datadog's host maps and log processing capabilities help visualize server resource utilization and streamline troubleshooting by grouping logs into patterns, making it easier to identify trends and anomalies. Users are encouraged to start monitoring Vault with Datadog by signing up for a free trial to experience its full suite of features.

Vault is a tool for managing, storing, and securing access to secrets, such as encryption keys, credentials, certificates, and tokens. It provides an application identity-centric approach, authenticating clients against trusted sources of identity before granting them access to data. Vault supports various storage backends, including etcd, Amazon S3, Cassandra, and Consul, and can be deployed on platforms like Kubernetes and AWS. To ensure optimal performance, it's essential to monitor key metrics such as core metrics, usage metrics, storage backend metrics, audit device metrics, resource usage metrics, replication metrics, and logs. Correlating these metrics with alerts can help identify potential security risks and ensure the overall health of the Vault cluster.

Datadog provides an Android SDK that helps developers collect, analyze and resolve errors and fatal crashes in their mobile applications. The SDK collects crash data from Android devices, transforms it for analysis, surfaces the most critical issues, creates intelligent alerts to notify users of a crash, and enriches crash data with meaningful metadata. With Datadog's Real User Monitoring (RUM) Android SDK, developers can identify the root cause of crashes, prioritize them, and create alerts to instantly notify them of significant issues. The platform provides built-in RUM dashboards and Error Tracking features that enable developers to visualize all types of crash data and quickly triage and respond to the most urgent application issues.

HashiCorp Vault provides a suite of tools for monitoring and managing its deployment, including querying its HTTP APIs for telemetry data, using the `vault debug` command for comprehensive data collection, and interacting with its web-based UI for basic configuration tasks. The `/sys/health` and `/sys/replication` endpoints are particularly useful for assessing server health and replication status, providing insights into data encryption and synchronization processes. The `vault debug` command offers extensive data collection capabilities, including metrics and profiling data, stored in a single archive for ease of analysis. Vault's UI, while user-friendly for simple configurations, lacks the advanced monitoring capabilities of third-party solutions. For more detailed analysis, Vault can forward its telemetry data to external platforms like Datadog, which enhances long-term trend analysis by integrating metrics, logs, and alerts from Vault with broader infrastructure data. Server and audit logs are critical for troubleshooting and security, with server logs detailing operational processes and audit logs documenting client interactions for security oversight. Overall, while Vault's built-in tools offer immediate insights, comprehensive monitoring is best achieved by leveraging third-party integrations.

Gartner published the 2021 Magic Quadrant for Application Performance Monitoring (APM), positioning vendors based on their ability to execute and vision completeness. Datadog, a unified platform used by customers worldwide, moved from "Visionary" to "Leader" quadrant due to its robust feature set and customer feedback. Its APM tools help users quickly identify and remediate system bottlenecks or errors for optimal user experience. Gartner's research publications do not endorse vendors, products, or services and disclaims all warranties.

Datadog has been positioned as a "Leader" in the 2021 Magic Quadrant for Application Performance Monitoring by Gartner, up from its previous "Visionary" distinction. This recognition is based on Datadog's ability to execute and the completeness of its vision, with customers worldwide using its unified platform to eliminate visibility gaps and optimize their applications for better user experience. The company's APM feature set provides end-to-end distributed tracing, seamless correlation of traces with full-stack telemetry data, and a simple setup process, empowering users to quickly identify and remediate bottlenecks or errors in their system. Customer feedback plays an important role in shaping the report, and Gartner disclaims all warranties regarding its research publications.

Datadog Cloud SIEM's Security Rules allow you to define patterns that match specific activity from an attacker as dangerous and alert you when these patterns occur. The new value detection method enables the system to learn historical behavior of specified entities within your environment, notifying you whenever unfamiliar values appear in attribute fields of their associated logs. This feature monitors behavioral patterns in your logs and automatically notifies you whenever a deviation from the baseline occurs. New value-based rules analyze users and entities over a chosen period of time, generating Security Signals for anomalies such as requests coming from new countries or IP addresses. By setting up detection rules with accurate training windows, you can proactively detect unknown threats before they become serious issues.

Datadog has announced the acquisition of Sqreen, an application security platform, to enhance its Cloud SIEM offering and provide a comprehensive security solution for modern DevSecOps practices. Sqreen's Runtime Application Self-Protection (RASP) and In-App Web Application Firewall (WAF) protect distributed applications against sophisticated attacks that target application logic. The integration of Sqreen's technology with Datadog's observability capabilities will enable customers to have a unified monitoring and security platform, leading to fewer agent deployments, simpler configurations, and better collaboration among development, security, and ops teams.

Sqreen, an application security platform, is joining the Datadog team to provide a robust security monitoring solution for cloud environments. Sqreen's Runtime Application Self-Protection (RASP) and In-App Web Application Firewall (WAF) protect modern distributed applications against sophisticated attacks. The integration of Sqreen with Datadog's Cloud SIEM will offer a comprehensive security strategy, combining observability capabilities with active application protection. This partnership aims to simplify configurations, reduce agent deployments, and enable seamless collaboration between security engineers, developers, and SREs.

Azure Service Health is a feature that notifies users about issues affecting their environment's availability, such as service incidents, planned maintenance periods, or regional outages. Datadog has recently enhanced its Azure integration to support monitoring of Service Health issues, allowing users to keep track of the health of their Azure environment and take proactive measures to mitigate downtime. With this integration, rich, contextual Service Health events appear within the event stream, where they can be monitored and correlated with data from over 650 infrastructure technologies. Datadog automatically converts metadata from the Azure Events API into key:value tags for easy filtering and searching of events. Users can also set up event monitors to get notified when specific types of Azure Service Health issues occur, and use Service Health issues to enrich their dashboards by overlaying them on graphs for better context and troubleshooting.

Azure Service Health is a feature that continuously notifies users of potential issues affecting their environment, such as service incidents, planned maintenance periods, or regional outages. Datadog has enhanced its Azure integration to include support for monitoring Service Health issues, enabling users to monitor and mitigate downtime proactively. With Datadog's integration, users can see rich, contextual Service Health events in their event stream, correlated with data from over 850 infrastructure technologies, including other Azure services. Users can slice and dice these events using tags, creating a cohesive view of Service Health issues and enabling them to track progress and take proactive measures. Additionally, Datadog provides the ability to set up event monitors to notify users when specific types of Service Health issues occur, and use Service Health events to enrich dashboards with context and helpful information.