March 2023 Summaries

Datadog's CoScreen is an innovative remote collaboration tool that combines interactive screen sharing and video conferencing, closely mimicking in-person collaboration. It enables engineering and product teams to simultaneously video chat, share multiple application windows, and seamlessly interact across each other’s desktop environments, making collaborative software engineering and product development fluid and intuitive. CoScreen's latest features and improvements enhance remote pair programming sessions by enabling quick and seamless initiation of meetings from various platforms, facilitating smooth collaboration with frictionless window management, and preventing accidental changes in shared windows.

Datadog CoScreen is a remote collaboration tool that enables seamless pair programming and real-time video conferencing, allowing engineering and product teams to collaborate fluidly across desktop environments. It combines interactive screen sharing and video conferencing in a way that mimics in-person collaboration, making collaborative software engineering and product development intuitive and productive. The new v5 release introduces features such as persistent virtual workspaces, improved window management, and integrations with Datadog Incident Management, enabling teams to quickly initiate pair programming sessions, troubleshoot issues, and collaborate smoothly without context switching or interruptions. With UI improvements and integrations, CoScreen accelerates pair programming, facilitating development, debugging, code reviews, incident response, technical onboarding, and more.

Azure Arc is a service that extends the Azure platform to help organizations build cloud-native applications, secure Windows and Linux servers, and deploy data services across on-prem data centers and other clouds. Datadog has introduced an integration with Azure Arc to provide insights into Arc environments and simplify hybrid and multi-cloud management. The integration allows users to monitor the connection status of their Arc-connected clusters and servers, add Azure Arc tags onto host monitoring data for identification and troubleshooting, identify Arc-enabled servers that have the Datadog Agent installed, and deploy the Datadog Agent as an extension onto Arc-enabled servers. This helps organizations maintain observability across their hybrid infrastructure and manage resources effectively in a multi-cloud environment.

Azure Arc is a service that extends the Azure platform to help organizations build cloud-native applications, secure servers, and deploy data services across on-prem data centers and other clouds. The new integration with Datadog provides insights into Arc environments, allowing users to monitor connection status, identify issues, and deploy the Datadog Agent as an extension onto Arc-enabled servers. With this integration, users can visualize host status, set up monitors for alerts, and create custom dashboards and monitors using Azure tags. This enables hybrid and multi-cloud management, simplifying observability across environments, and aiding in migrations and hybrid management.

Calico is a versatile networking and security solution that supports various technologies such as Iptables, eBPF, Host Network Service (HNS for Windows), and Vector Packet Processing (VPP) for containers, virtual machines, and bare-metal workloads. Datadog's Calico integration provides granular detail into traffic between Kubernetes resources and other workloads, enabling users to ensure that their network policies are properly filtering traffic via iptables. The integration allows tracking of network policies, restricting traffic to host endpoints and workload endpoints, monitoring errors in iptables and ipsets, leveraging Datadog's Kubernetes integration for rich context around Calico performance, and getting real-time visibility into network policies and endpoints.

Calico is a versatile networking and security solution that features a plugable dataplane architecture, supporting various technologies including Iptables, eBPF, Host Network Service (HNS for Windows), and Vector Packet Processing (VPP) for containers, virtual machines, and bare-metal workloads. Datadog's Calico integration provides granular detail into traffic between Kubernetes resources and other workloads, enabling users to ensure that their network policies are properly filtering traffic. The integration allows users to track network policies, restrict traffic to host endpoints and workload endpoints, monitor errors in iptables and ipsets, leverage Datadog's Kubernetes integration to get rich context around Calico performance, and monitor dataplane and ipsets logs. By using the integration, users can prevent connectivity issues from disrupting their end-user experience, secure their workloads from malicious actors when sensitive data has been compromised, and troubleshoot issues more effectively.

SQL Server AlwaysOn availability groups provide high-availability support and streamline automatic failovers and disaster recovery. However, they can be complex to manage due to multiple points of failure in each cluster. Datadog's AlwaysOn view helps users monitor the state of their nodes and prepare for possible failovers with color-coded visualizations and historical data. It also enables users to analyze historical metrics to investigate cluster bottlenecks and failures, allowing them to spot issues and improve infrastructure support for databases.

SQL Server AlwaysOn availability groups provide database clusters that streamline automatic failovers and disaster recovery, but can be complex to understand and troubleshoot. Datadog's AlwaysOn view in Database Monitoring provides high-level overviews of these groups with color-coded visualizations, historical data for each node, and out-of-the-box timeseries graphs for log, redo, and secondary lag time metrics. This enables users to quickly assess database health, prepare for failovers, analyze historical metrics to investigate cluster bottlenecks and failures, and set up monitors to alert them when nodes fall out of sync or exhibit unusual behavior.

The Datadog Cloud SIEM Investigator for Google Cloud is a newly announced tool that enhances visibility for DevOps and security teams within Google Cloud environments, complementing the existing AWS support and soon to include Microsoft Azure. It utilizes Google Cloud Audit Logs to visualize activities in resources like Google Cloud Storage and Google Compute Engine, allowing teams to correlate this information with service accounts and user identities. This tool aids in identifying potential security risks by providing detailed insights into user interactions and operations performed on specific resources, which is crucial for determining the legitimacy of activities such as account creation and permission grants. By integrating with Log Explorer and Security Signals, it fosters better collaboration between DevOps and security teams in investigating flagged events and logs. The Investigator's schematic mapping of activities helps in distinguishing routine actions from potential threats, thereby improving response times and the effectiveness of security investigations.

Migrating an on-prem database to a public cloud like Azure can offer benefits such as reduced maintenance costs and improved scalability. However, the process can be complex and challenging. To navigate this migration effectively, organizations need to adopt a suitable Azure solution for their workloads, benchmark current SQL Server performance, and monitor their Azure-hosted SQL workloads using tools like Datadog. By following these steps, businesses can ensure a successful transition to the cloud while maintaining consistent performance levels.

The TeamCity integration with Datadog has been revamped to provide enhanced visibility into CI/CD pipelines, enabling users to monitor pipeline health, detect issues, and eliminate bottlenecks. With new metrics such as failed build count, job duration trends, and the number of builds at different stages in the pipeline, users can gain a holistic view of their system and troubleshoot pipeline problems more effectively. The integration also allows for setting custom monitors on TeamCity pipelines to alert users when specific issues arise. By leveraging deeper visibility into build pipelines and system health metrics, users can identify bottlenecks within their pipelines over time and improve the efficiency of their CI/CD workflows.

Migrating an on-prem database to a public cloud like Azure offers several benefits, including reduced infrastructure management needs, dynamic scaling, disaster recovery capabilities, and cost savings. However, the migration process can be complex and daunting. Choosing the right Azure-managed SQL solution is crucial, with three options available: SQL Server on Azure Virtual Machines (VMs), Azure SQL Managed Instance, and Azure SQL Database. Each solution has its unique benefits, such as direct OS access for VMs or managed platform features like automated backups and software patches for Managed Instance. Establishing benchmarks for current SQL Server performance is essential to determine the best solution and plan a successful migration. Monitoring tools like Datadog are vital for effective benchmarking, providing visibility into key health and performance metrics to ensure workloads remain consistent or improve after migration. By leveraging Datadog's integration with Azure, organizations can gain complete visibility into their cloud-hosted SQL workloads, making it easier to identify trends, set expectations, and optimize resource utilization.

The text discusses the enhancements made to Datadog's integration with TeamCity, a CI/CD server, which now offers improved visibility and control over build pipelines. The integration provides new metrics, such as failed build counts and job duration trends, which are displayed on a comprehensive dashboard that includes alerts and logs to help troubleshoot issues effectively. The narrative includes hypothetical scenarios to illustrate how engineers and site reliability engineers can use these tools to detect and resolve issues, eliminate bottlenecks, and improve build performance. Additionally, the text highlights the ability to correlate build events and logs with performance metrics, providing deeper insights into the health of CI/CD pipelines. It concludes by inviting new users to experience the benefits of the integration through a free trial of Datadog.

Kubernetes has the potential to help organizations use resources more efficiently, but in practice, they don't always lead to cost savings. This is due to how resources are allocated in Kubernetes environments and the fact that many organizations pay for resources they don’t end up using. To mitigate this issue, teams need to request the right amount of CPU and memory requests for their containers (i.e., size their pods correctly). Tools like the Kubernetes Vertical Pod Autoscaler and historical data can be used to rightsize workloads. Additionally, it is important to understand how CPU and memory limits affect scheduling and performance in Kubernetes environments.

The Kubernetes scheduler plays a crucial role in allocating resources to pods in a cluster. However, if not properly sized, containers may end up using more resources than needed, leading to increased cloud bills and potentially even out-of-memory conditions. To mitigate this issue, FinOps professionals can follow practical tips such as estimating resource requirements based on application code and benchmarking, optimizing resource allocation with tools like the Vertical Pod Autoscaler, and utilizing historical data from Datadog to rightsize workloads. Additionally, setting limits for CPU and memory requests can help ensure better predictability and avoid wasting resources. Ultimately, it is essential to take into account the nature of your workloads and business needs when setting resource requests and limits.

RabbitMQ is a message broker that facilitates communication between different parts of an application by routing messages between producers and consumers. It is well-suited for microservices architectures, as it allows loose coupling between services. Key components within RabbitMQ include exchanges, queues, and bindings, which define the logic of routing messages. Monitoring RabbitMQ involves tracking metrics related to these components, such as message rates, queue depths, consumer utilization, and resource usage (e.g., file descriptors, network sockets, disk space, memory). By monitoring these metrics, users can ensure that their messaging setup is working efficiently and effectively.

RabbitMQ provides several built-in monitoring tools and plugins to help users collect metrics about their messaging setup. These include the CLI tool rabbitmqctl, which provides a quick snapshot of key metrics; the management plugin, which extends the host node with a web server that reports metrics via UI and API; and the Prometheus plugin, which enables data collection in the OpenMetrics format. RabbitMQ also offers plugins for tracing events and messages within the system, such as the event exchange, firehose, and rabbitmq_tracing tool. By leveraging these tools, users can gain valuable insights into the performance of their RabbitMQ applications and identify potential bottlenecks or issues.

In Part 2 of the series, RabbitMQ's built-in tools and plugins are discussed for monitoring various aspects of an application such as message traffic handling, node memory consumption, consumer operation status, etc. However, to gain a comprehensive understanding of applications, it is essential to see how RabbitMQ performance relates to the rest of the stack. Datadog provides an all-at-once view of key RabbitMQ metrics with its RabbitMQ dashboard and allows setting alerts for notifying when the availability of messaging setup is at risk. To set up comprehensive monitoring using Datadog's RabbitMQ integration, first install the Agent on your host which checks for RabbitMQ performance metrics and sends them to Datadog. The Agent can also capture metrics and trace requests from other systems running on your hosts. Integration with RabbitMQ can be done either through the management plugin or the Prometheus plugin. Once the integration is set up, RabbitMQ should start reporting metrics, events, and service checks to Datadog. The RabbitMQ dashboard comes in two versions - the management dashboard and the Prometheus dashboard, depending on which plugin was used during installation. Both dashboards provide information about node status for optimizing performance, including open/closed channels, active consumers, memory utilization, etc. Datadog also allows setting alerts to notify your team of performance issues related to RabbitMQ's resource usage such as memory or disk use. Additionally, distributed tracing and APM can be used to understand the role of RabbitMQ within applications and how often it handles traffic from HTTP servers.

The Datadog Service Catalog provides a unified view into service ownership, dependencies, performance, reliability, and security posture, helping improve collaboration during incidents, detect monitoring gaps, and promote observability best practices. The new setup experience makes it easier for users to add service metadata without leaving the Datadog application, allowing them to track the source of each service's metadata and avoid unintended updates. Additionally, the Service Catalog enables auto-generated code to make metadata simpler to manage, leverage GitHub integration to automate service management, and monitor latest updates to ensure clarity and effectiveness in troubleshooting and delivering value to end users.

Modern applications generate large volumes of logs daily, providing valuable insights into system states and service performance. However, effectively analyzing root causes requires viewing these logs within the context of complex, distributed systems. To address this challenge, Datadog has developed Transaction Queries, which simplify log aggregation based on shared attributes to expose context and calculate key metrics for specific processes or user journeys. This feature enables users to create end-to-end context from their logs, harnessing them for efficient troubleshooting across various use cases such as e-commerce data, web user activity, financial transactions, authentication sessions, and CI/CD pipelines. By grouping logs by primary identifier values and optional boundary conditions, users can quickly identify problematic areas and investigate the root causes of issues within their applications.

RUM Deployment Tracking, a feature from Datadog, offers comprehensive monitoring and troubleshooting capabilities for developers and product teams to manage frontend deployments effectively. This tool provides out-of-the-box performance metrics and Powerpacks, enabling users to easily track performance issues, compare different versions, and assess the impact of new features on app performance. By consolidating deployment data into a single overview, RUM Deployment Tracking helps identify the root causes of errors, performance regressions, and user experience degradations. Additionally, it facilitates the creation of custom dashboards to monitor key health metrics and business impacts of updates in real-time. Users can access detailed visualizations, relevant session replays, and error messages for efficient troubleshooting, ultimately ensuring safe and reliable code releases.

Delivery Hero, a leading local delivery platform, has successfully managed to scale its operation during the pandemic by leveraging Kubernetes clusters running on Amazon EKS. However, operating shared Kubernetes clusters made it difficult for them to fully understand their cloud costs. To address this issue, they used custom Datadog dashboards that combined Kubernetes usage metrics with detailed cost data from Kubecost. They also utilized memory and CPU request recommendations from Vertical Pod Autoscaler (VPA) to evaluate and revise their clusters' resource allocations. This approach has led to a 10 percent decrease in Delivery Hero's cloud costs over 48 days, with an ongoing goal of reducing overall cloud costs by 30 percent.

The text discusses the challenges of blocked queries in database management, emphasizing the role of Datadog Database Monitoring in identifying and resolving these issues. Blocked queries, often caused by inefficient designs or resource saturation, can result in increased latency and user frustration. The Datadog tool offers in-depth visibility into root blocking queries, helping users quickly troubleshoot and optimize database performance. It allows users to assess blocked active connections, understand the root blockers, and examine query performance over time through features like Blocking Summaries and the Active Connections tab. These functionalities facilitate precise identification of blocking activities, aiding in the resolution of performance bottlenecks. By breaking down wait events into comprehensible categories, Datadog enhances understanding and accelerates troubleshooting, ultimately supporting effective database performance management and timely incident resolution.