June 2020 Summaries

Datadog introduces automated browser tests that allow users to monitor user experience alongside metrics, distributed traces, and logs from their applications and infrastructure. The new feature enables teams to automate their user experience monitoring and ensure users can complete actions like signing up for a new account or adding items to a cart. With the help of machine learning, Datadog's browser tests detect changes in an application and update tests accordingly. Users can create tests in minutes using the Web Recorder without any knowledge of specialized frameworks or coding skills. The platform also offers fully hosted browser tests with global locations and end-to-end visibility for troubleshooting.

Datadog has introduced Synthetic Monitoring, a new feature that enhances visibility on the platform by monitoring applications, API endpoints, and key business flows through simulated user requests. This helps ensure uptime, identify regional issues, track application performance, and manage SLAs and SLOs. Synthetic tests provide end-to-end visibility for validating critical business transactions and can be run from Datadog-managed locations globally or custom locations that matter to the business. The integration with other Datadog features allows users to monitor results of Synthetic tests alongside application performance metrics and infrastructure metrics, minimizing alert fatigue and providing a comprehensive view of user-facing performance in one place.

Collaboration and communication are crucial for successful implementation of service level objectives (SLOs). Datadog simplifies cross-team collaboration by enabling everyone to track, manage, and monitor the status of all their SLOs in one place. Choosing the best SLO depends on whether you're using time-based or count-based data to calculate the SLI. Monitor-based SLO tracks time-based data while metric-based SLO uses your metrics in Datadog to calculate its SLI. Names, descriptions, and tags help keep SLOs organized and streamline communication across teams. Grouping SLOs with tags enables tracking of individual clusters or availability zones in context with the overall status. Enhancing dashboards with SLOs helps visualize key performance metrics from infrastructure and applications.

Service level objectives (SLOs) have become an essential part of site reliability engineering (SRE) practices, with Google pioneering best practices in this area. SLOs help organizations balance product development and operational work to ensure a positive end-user experience. In this three-part series, the fundamentals of SLOs, how to manage them using Datadog, and how to maximize their value will be discussed. Key terms related to SLOs include Service Level Indicators (SLIs), which measure service levels provided to end users; Service Level Objectives (SLOs), the targeted levels of service measured by SLIs; Service Level Agreements (SLAs), contractual agreements outlining expected service levels with consequences for non-compliance; and error budgets, acceptable unreliability levels. Setting SLOs involves considering end users' expectations, developers' priorities, and operations engineers' goals. Developers aim to add features, while ops engineers maintain stability. SLOs help align these teams by setting reliability targets and allowing them to objectively decide which projects or initiatives to prioritize based on error budgets. To create useful SLOs, organizations should understand how users interact with their applications, identify critical user journeys, and select appropriate SLIs within categories such as response/request, storage, and data pipeline. Good SLIs directly affect user satisfaction and are typically measured using data from components closest to the user. SLOs are created by setting a target value or range of values for an SLI over a specified period. Realistic targets should be set, with typical industry standards being expressed as a number of nines (e.g., 99.9 percent). Experimentation and refinement of SLOs is encouraged to find the most optimal values. In summary, picking the right SLIs and transforming them into well-defined SLOs can help organizations improve feature velocity and system reliability by focusing on what's truly important for end users.

This is a guide on Service Level Objectives (SLOs) and how to use them for service reliability engineering. SLOs are rooted in the idea that service reliability and user happiness go hand in hand, and setting concrete and measurable targets helps strike the right balance between product development and operational work. The key concepts include Service Level Indicators (SLIs), which measure the level of service provided to end users, and Service Level Objectives (SLOs), which are targeted levels of service measured by SLIs. SLOs also help teams prioritize projects and align their goals with those of the business. To get started with SLOs, you need to identify key stakeholders, understand user experience, select useful SLIs, set realistic targets, experiment with different values, and avoid overcomplication. By following these steps, organizations can create well-defined SLOs that improve feature velocity and system reliability.

Datadog simplifies cross-team collaboration by enabling everyone in an organization to track, manage, and monitor the status of all their service level objectives (SLOs) and error budgets in one place. To effectively manage SLOs, teams need to evaluate the impact of their work against established service reliability targets to improve end-user experience. The best approach depends on the use case, but Datadog offers three types of SLOs: metric-based, time slice, and monitor-based. Teams can visualize their SLOs alongside relevant services and infrastructure components on dashboards and share real-time status with stakeholders. To ensure accurate SLO status information, teams can use SLO status corrections to exclude data from calculations. Effective naming and tagging strategies are crucial for streamlining communication and keeping SLOs organized. Datadog's Saved Views and group-based visualization enable teams to quickly find and track their most frequently used SLOs. By enhancing dashboards with SLOs, teams can gain a high-level summary view of their SLOs by grouping and proactively monitor the status of their SLOs with automatic alerts.

HiveMQ is an open-source MQTT broker designed for enterprise-scale IoT environments, facilitating reliable and secure data transfer between connected devices and downstream applications. The integration with Datadog enhances monitoring capabilities by providing full observability of HiveMQ brokers, allowing users to collect and visualize key data such as client connection activity and message throughput. This integration offers a customizable dashboard for real-time visibility into connection health, broker load, and backend service performance. It enables users to troubleshoot client connection fluctuations by monitoring connection metrics and identifying potential issues through event logs. Additionally, Datadog aids in monitoring message throughput, helping identify problems such as ungraceful disconnects and dropped messages due to reasons like "Memory Exceeded." By collecting extensive system resource metrics and integrating with over 400 technologies, Datadog ensures comprehensive insight into the entire IoT environment, allowing for efficient identification of resource bottlenecks and troubleshooting of potential issues.

The guide discusses the importance of end-to-end (E2E) testing in software development and provides best practices for creating efficient and meaningful test suites. It emphasizes the shift-left model, where tests are integrated into the development cycle earlier to find and fix issues faster. The guide covers key aspects of building E2E tests, including defining test coverage, breaking down workflows with smaller tests, creating meaningful assertions, and designing coherent test suites. Additionally, it highlights the importance of maintaining the state of applications with idempotent tests, sending notifications that enable teams to respond faster, and organizing tests by location, device, and other metadata.

Logs are valuable sources of information that can be used for root cause analysis and audits. However, it is not always necessary to view the granular details of every log, especially in dynamic environments generating large volumes of logs. Instead, performing analytics on logs in aggregate can provide deep insights into applications and infrastructure at a lower cost. Datadog's log processing pipelines can automatically enrich logs with attributes that categorize them as they are ingested, allowing for efficient analysis of all logs. These pipelines can help categorize logs based on URL patterns, response duration, and standard severity levels. Once the logs are categorized, they can be used to drive indexing policies and analyze trends across different categories of logs.

Datadog's log processing pipelines can automatically enrich logs with attributes that categorize them as they are ingested, enabling efficient analysis of applications and infrastructure at lower cost. Logs can be categorized based on URL patterns, response duration, or standard severity levels, allowing for the creation of custom categories, indexing policies, and metrics. These categories enable the exclusion of low-priority logs while archiving all enriched logs in long-term cloud storage. The use of log categories also facilitates the analysis of trends across all categories of logs, providing critical visibility into high-priority logs and long-term trends without the expense of indexing and retaining everything.

RethinkDB is an open-source document-oriented database that enables real-time updates and easy sharding and replication. Datadog's RethinkDB integration provides granular visibility into the health and performance of your RethinkDB deployment, allowing you to monitor client activity, track scaling operations, ensure availability, and more. With this integration, users can customize dashboards, set automated alerts for server issues, and gain insights into how clients interact with their database cluster.

Datadog Synthetic Monitoring introduces browser tests to simulate key user journeys and validate business-critical transactions. The platform enables teams to proactively monitor their applications, ensuring seamless file uploads and downloads. With automated user experience monitoring, users can create tests without coding knowledge, and the system updates tests automatically when changes are detected in the application's UI. Integration with collaboration tools such as Slack and PagerDuty allows for quick investigation of issues. Datadog Synthetic Monitoring provides a complete picture of an application's performance by integrating frontend and backend metrics, traces, and logs.

Datadog Synthetic Monitoring enables organizations to proactively monitor their application's user experience, ensuring that users can seamlessly complete business-critical transactions, such as uploading and downloading files. This is achieved through automated browser tests that simulate key user journeys, providing a user's view of the application and allowing teams to quickly investigate issues without switching contexts. The solution is self-maintaining, automatically updating tests when it detects changes to the application's UI, freeing up teams to focus on building new features. By integrating with collaboration tools like Slack and PagerDuty, teams can loop in the right teams to start investigating issues immediately. Datadog Synthetic Monitoring provides rich context for troubleshooting application issues, regardless of whether they stem from the frontend or backend, helping organizations ensure their application is fully functional and maintain a high level of user experience.

Datadog has integrated its platform with RethinkDB, a document-oriented database that enables real-time updates and sharding capabilities. The integration provides granular visibility into database servers, tables, and shards, helping ensure the health and performance of RethinkDB deployments. It automatically queries system statistics tables to track client activity within the deployment, providing detailed information on the current number of clients and connections. The integration also tracks replication operations, server availability, and database monitoring metrics, enabling users to identify potential issues and optimize their cluster for better performance. With this integration, users can collect RethinkDB metrics and logs, as well as instrument their applications with Datadog APM, to gain comprehensive visibility into the health and performance of their database cluster.

Creating security policies for network-connected devices is crucial for safeguarding company data, particularly in environments where personal devices are part of the network. Carbon Black offers endpoint security software to monitor device safety and enforce security policies, which becomes even more effective when integrated with Datadog. This integration allows real-time monitoring of endpoint activity and potential threats, including malicious software, by collecting CB Defense audit logs. Users can create custom dashboards and threat detection rules to enhance network visibility, with Datadog offering control over log indexing and archival. Carbon Black assigns severity scores to detected threats, enabling prioritized responses, while Datadog facilitates isolation and repair of compromised devices by providing detailed context through logs. This collaboration with Datadog Cloud SIEM and its extensive integration capabilities offers a comprehensive platform for vulnerability detection and infrastructure monitoring.

Datadog's annual conference, Dash 2020, will be a free virtual event taking place on Tuesday, August 11. The event aims to bring together the global community around common goals of enhancing teams, organizations, and infrastructure performance, reliability, and scale. Attendees can expect breakout sessions, new product releases, workshops, and opportunities for networking through video sessions with engineers, product managers, partners, and other attendees. Last year's Dash included talks from industry leaders, hands-on interactive workshops, and exclusive after-event activities. Registration is open for the online event, and virtual workshops will be hosted in the weeks following Dash 2020 with a portion of proceeds going toward COVID-19 relief efforts.

Datadog has simplified its AWS integration process by introducing a 1-click installation feature that automates the configuration of monitoring AWS services like EC2 and Lambda. The new integration also includes the Datadog Forwarder, which gathers serverless metrics, logs, and traces from AWS services for analysis and alerting purposes. Additionally, it provides quick visibility into other AWS services such as RDS, Elasticache, and more. With this 1-click integration, users can start monitoring their AWS environment in just a few minutes.

Datadog has simplified the process for installing its AWS integration, allowing users to get started in minutes. The new 1-click installation automates configuration using CloudFormation, making it easier to gather metrics, logs, and traces from AWS services like EC2 and Lambda. This integration also includes the Datadog Forwarder, a Lambda function that brings serverless metrics, logs, and traces into Datadog. With this setup, users can visualize serverless visibility, collect custom metrics, enable Lambda enhanced metrics, and create alerts based on any of their Lambda metrics. The AWS 1-click integration also provides quick visibility into the wider AWS landscape, including RDS, EC2, Elasticache, and more. Users can view built-in dashboards for many services, gain instant insight into their AWS environment, and monitor dozens of AWS services plus over 850 other technologies in just a few minutes.