August 2020 Summaries

Amazon's Bottlerocket is an open-source Linux-based operating system designed for containers and optimized to only include essential software. It allows seamless updates and rollbacks, and can be customized according to specific needs. Datadog offers real-time visibility into container activity on Bottlerocket, whether running on-prem or using an orchestrator like EKS. The Datadog Agent collects metrics, traces, logs, and more from containers and underlying hosts for visualization, search, and analysis in Datadog. It provides customizable dashboards, real-time monitoring of container activity, and integration with the full AWS ecosystem.

Amazon's Bottlerocket is a Linux-based open-source operating system designed with containers in mind, optimized and stripped down to only the essential software needed to run containers. It offers streamlined updates, an open-source model, and efficient resource utilization ideal for any containerized environment. The Datadog Agent provides real-time insights into containers running on Bottlerocket, offering customizable dashboards, a Live Container View, and support for over 400 integrations, including the full AWS ecosystem. With Datadog, users can get actionable information to troubleshoot cluster problems, track changes in their containerized applications, and monitor their entire cloud stack from a single pane of glass.

Datadog has achieved Moderate-Impact authorization from the Federal Risk and Authorization Management Program (FedRAMP) to safely and securely monitor infrastructure and workloads running in AWS GovCloud (US) environments. Datadog provides complete visibility into these environments, including monitoring of host health, latency distribution, memory allocation, error rates, endpoint performance, and more. The platform integrates with the full suite of AWS services, such as Amazon EC2, AWS Lambda, and Amazon S3, allowing users to gather business-critical metrics and spot potential issues. Datadog also simplifies inventory management by providing real-time insight into cloud asset posture and metric correlation detection tools. Additionally, the platform offers unified visibility into entire IT infrastructure, including network device monitoring, database monitoring, and cloud security management, making it an ideal solution for government agencies and public-sector organizations.

Datadog enhances Kubernetes monitoring by providing comprehensive visibility into clusters, pods, and other resources, enabling efficient troubleshooting and performance optimization. The Live Containers view now offers curated insights into Kubernetes workloads, allowing users to monitor state and resource specifications, correlate node activity with logs, and filter resources using tags and labels. The platform's interface is integrated with the rest of Datadog, offering detailed context for each Kubernetes resource, including the ability to drill down into individual pods and view related events, metrics, and configurations. Additionally, Datadog's Cluster Map provides a high-level overview of the entire Kubernetes environment, highlighting issues such as resource misconfigurations. The platform also includes Kubernetes Overview dashboards for monitoring critical performance data, facilitating the identification and resolution of resource usage issues. Users can access these features to maintain optimal performance of their Kubernetes applications, with the option to sign up for a free trial if they do not already have a Datadog account.

At Dash 2020, Datadog announced several new features and products aimed at enhancing visibility into distributed applications and infrastructure for engineering teams. These include Incident Management, a mobile app, Cloud Security Posture Management (CSPM), Continuous Profiler, Synthetic CI/CD Testing, Error Tracking, Mobile Real User Monitoring (RUM), APM Live Search, Marketplace, Recommended Monitors, and BigPanda integration. Additionally, Datadog now integrates with AWS Serverless Application Model (SAM) and AWS Cloud Developer Kit (CDK) for zero-instrumentation serverless observability.

The Datadog Partner Network (DPN) has launched the Datadog Marketplace, a digital marketplace for developers to trade third-party applications with other Datadog users. This marketplace offers custom integrations and expanded functionality from partners in the cloud service industry. It includes strict vetting standards and a fully managed billing system. The Marketplace provides detailed information on each available app, including pricing, installation instructions, data collection specifications, and customer support contact details. Paid applications can be trialled for 14 days before charges are applied to users' accounts. Datadog partners can publish their applications in the marketplace and gain access to dedicated sales and marketing resources, as well as premium product training materials. The Marketplace aims to expand monitoring practices and build new tools on the Datadog platform while increasing brand awareness for its partners.

Datadog has released Mobile Real User Monitoring (RUM) to help mobile developers improve the performance of their applications and troubleshoot errors and crashes. By connecting client-side and server-side metrics, traces, and logs, Mobile RUM provides full visibility into the performance of mobile applications. The tool allows users to monitor key performance metrics in real-time, investigate individual user sessions, set up alerts for crashes and errors, and collect analytics on user behavior. This helps developers deliver a seamless user experience and make better decisions around performance and design optimizations.

Consul is a distributed configuration and service-discovery tool that enables dynamic service discovery and reacting to real-time changes in service availability. It uses a distributed client-server architecture based on the Raft consensus algorithm, providing fault-tolerance with strong data consistency. Datadog integration allows monitoring of Consul health and availability across clusters. Key features include built-in health checks for both node and service levels, latency metrics to track leader and follower interactions, cluster health metrics, event monitoring, tagging for scalable monitoring solutions, and custom alerts based on metric thresholds.

The new feature of Datadog Error Tracking allows developers to automatically group application errors into issues, providing more context and enabling better root cause analysis. This feature is available for frontend JavaScript as well as Android applications and all languages supported by Datadog with distributed tracing and APM. It extracts error messages from Real User Monitoring (RUM) and APM data, eliminating the need to configure an SDK or modify application code. The feature provides actionable insights into application errors, making it easier to troubleshoot issues that affect users most. It also offers more revealing stack traces for frontend JavaScript, reversing minification processes using source maps, and supports .dSYM parsing for mobile applications. Additionally, Error Tracking builds on existing Datadog data, requiring no additional setup, and provides a workflow state (open or ignored) to help teams track the status of issues.

Datadog has unveiled new products and features that provide engineering teams with deeper visibility into their distributed applications and infrastructure. These additions include Incident Management, which allows teams to create incidents and track them from notification to resolution; a mobile app for on-call engineers to triage alerts on the go; Cloud Security Posture Management (CSPM), which helps monitor production environments for misconfigurations and maintain compliance with industry standards; APM & UX monitoring features such as Continuous Profiler, Synthetic CI/CD Testing, Error Tracking, Mobile Real User Monitoring, and APM Live Search. Additionally, Datadog has integrated with various third-party tools like BigPanda and Amazon Data Firehose to provide enhanced logging and security capabilities. The platform also offers a Marketplace for partners to build custom apps and integrations, and Recommended Monitors for smart alerting.

Datadog has announced the general availability of Cloud Security Posture Management (CSPM), which enables organizations to easily track the compliance posture of their production environment, automate audit evidence collection, and catch misconfigurations that leave them vulnerable to attacks. CSPM is built on top of Datadog's existing platform and provides a centralized observability solution for security teams to continuously check cloud resource configurations, detect drift from critical compliance controls, prioritize and remediate potential misconfigurations, and correlate findings with other security issues. With support for industry-standard compliance frameworks like PCI DSS, SOC 2, and CIS, CSPM helps developers and security engineers ensure safe and compliant configurations across multiple cloud platforms, consoles, and security tools, reducing the risk of data breaches and regulatory non-compliance.

Datadog has released Mobile Real User Monitoring (RUM), a tool that provides full visibility into the performance of mobile applications, allowing developers to deliver a seamless user experience. With Mobile RUM, users can improve application performance, effectively troubleshoot errors and crashes, understand how their users interact with the app, and deploy apps with confidence. The tool connects client-side and server-side metrics, traces, and logs, giving users an end-to-end view of their application's performance and enabling them to identify the root cause of issues quickly. Mobile RUM provides out-of-the-box dashboards, real-time insights into key performance metrics, errors, and resources, and allows users to customize dashboards to include unique business metrics.

The Datadog Partner Network has launched the Datadog Marketplace, a digital marketplace where developers can trade their third-party applications with other Datadog users. The platform introduces new ways for partners to increase the scope of their monitoring practices and build new tools on the Datadog platform, while expanding awareness of their brand. The Marketplace offers a one-stop integration shop, allowing users to access integrations from various technology providers in a simple installation workflow. Paid applications can be trialed for 14 days before charges are applied, and the platform is open for browsing and demoing partner applications. Membership in the Datadog Partner Network comes with benefits such as access to dedicated sales and marketing resources, premium product training materials, and more. The Marketplace turns Datadog into a development platform, opening up new possibilities for extending monitoring reach.

Datadog Synthetic CI/CD Testing integrates synthetic testing into the CI/CD pipeline, supporting shift-left testing to detect issues early and ensure robust deployments. This approach allows agile teams to incorporate existing Datadog Synthetic tests throughout the development process, facilitating continuous integration and deployment without slowing the workflow. It offers flexibility by enabling the same test suite to be customized for various environments, eliminating the need for separate tests for each stage. The tool also supports canary deployments and automatic rollback in case of failures, ensuring safe releases. Furthermore, the platform provides comprehensive troubleshooting capabilities by correlating test runs with application metrics and logs, aiding in efficient issue resolution. With integrations into popular CI tools and a user-friendly CLI, the system enhances cross-team collaboration and helps maintain a seamless user experience by preventing problematic code from reaching production.

Datadog has introduced "Recommended Monitors," a suite of preconfigured, customizable alert queries and thresholds for key infrastructure technologies like Kubernetes, Kafka, and Consul, designed to streamline the monitoring process and enhance system performance visibility. These monitors are developed in collaboration with technology partners and are vetted to be preventative, contextual, and actionable, allowing users to set up alerts quickly by selecting from the "Recommended Monitors" tab after installing the Datadog Agent. Each monitor includes default queries, alert thresholds, and tags that adhere to best practices, and they can be customized to suit specific needs. Users can incorporate these alerts into their workflows and link them with communication tools like Slack and PagerDuty for timely responses. Datadog also maintains public GitHub repositories where partners and community members can contribute and submit monitors, with submissions undergoing a rigorous review process to ensure reliability.

The text outlines the introduction and capabilities of Datadog's Cloud Security Misconfigurations, a tool designed to address governance, risk, and compliance challenges in cloud environments by automating the detection and remediation of cloud security misconfigurations. Misconfigurations are a major cause of data breaches, posing significant challenges for developer and operations teams trying to maintain secure and compliant cloud services. The tool supports industry-standard compliance frameworks such as PCI DSS, SOC 2, and CIS, and offers continuous monitoring of cloud resources, including AWS, Azure, Docker, and Kubernetes, with plans for further expansions. It provides expert-built dashboards for tracking compliance, customizable rules for evaluating controls, and the ability to correlate misconfigurations with other security issues to prioritize risk mitigation. The article emphasizes the importance of real-time visibility and the ability to quickly address misconfigurations to prevent security breaches, while also highlighting Datadog's integrations with collaboration tools like Slack and PagerDuty for efficient issue resolution.