February 2021 Summaries

Google Kubernetes Engine (GKE) Autopilot is now generally available and fully manages both the control plane and worker nodes to provide an even more hands-off experience for users. Datadog has partnered with Google Cloud to offer deep visibility into dynamic, containerized workloads on Autopilot. The integration allows users to monitor their applications running on GKE Autopilot by collecting metrics, traces, and logs from Kubernetes, Docker, and any of the 650+ integrations that they are using. Datadog's Live Containers feature delivers real-time insight into every layer of Kubernetes clusters, enabling users to surface performance issues and provide context for effective troubleshooting.

Google Cloud has announced the general availability of Google Kubernetes Engine (GKE) Autopilot, which manages both the control plane and worker nodes of a cluster. This allows users to focus on building their applications while leveraging Datadog's full feature set for comprehensive monitoring. GKE Autopilot provides fully managed infrastructure with opinionated best practice patterns, allowing for production-ready deployments and optimized resource usage. It also supports DaemonSets, node selectors, and workload scheduling constraints. Datadog integrates with GKE Autopilot to provide real-time visibility into containerized applications, including out-of-the-box dashboards, Live Containers, and custom dashboards. Users can track pod resource utilization, deployment rollouts, and node health, making it easier to troubleshoot performance issues and optimize costs.

Mary Jac Heuman and Miranda Kapin discuss the importance of tracking and aggregating data by region to enhance application monitoring, enabling visibility into errors, latency, and security threats. They introduce Datadog geomaps as a tool to visualize data on a color-coded world map, allowing users to quickly identify geographic patterns related to outages, app revenue, or request surges. Geomaps integrate with Datadog's RUM for frontend performance monitoring, using country ISO codes to group data and visualize it effectively. This feature aids in identifying regions with high latency, creating alerts, and making informed business and security decisions. The geomaps also facilitate mapping log data to detect security threats by recognizing unusual activity patterns, offering a way to track the origin of requests to private endpoints. Additionally, custom metrics derived from application logs can be visualized to gain insights into geographic trends, such as revenue distribution by country, thereby guiding strategic business decisions.

At Datadog, transitioning an existing job system to Kubernetes initially led to performance regression, with increased CPU time and slower job completion rates. The solution involved performance tuning, timing analysis, and optimizing Kubernetes configurations. Initial experiments showed underutilization and inefficiencies due to incorrect deployment setups and node configurations. By adjusting resource requests for pods and reducing the interval for worker status checks, the team reduced node count and improved efficiency. Key findings included that Kubernetes overhead was minimal in terms of memory and CPU, and further optimization could enhance performance. The transition ultimately enabled a more manageable and scalable system, leveraging Kubernetes' benefits across cloud providers.

The quick navigation menu feature of Datadog allows users to easily switch between various resources within the application environment for efficient troubleshooting and development workflows. Accessible through keyboard shortcuts, this menu provides shortcuts to access widget clipboard, create new dashboards, and navigate recent views. It also includes links to major features in Datadog and a search bar for quick locating of specific resources. This feature is now available across the entire app, enhancing monitoring workflows for users.

Datadog has introduced a quick navigation menu to help users quickly access key resources and streamline their monitoring workflows. This feature, accessible via the keyboard shortcut `cmd+K` on macOS or `ctrl+K` on Windows and Linux, provides shortcuts to the Widget Clipboard and dashboard creation, as well as links to major features in Datadog. The quick nav menu also surfaces recently visited views within Datadog, allowing users to instantly access their most frequently used resources. Additionally, a search bar at the top of the menu enables users to find specific pages or dashboards by searching for keywords related to their monitoring needs. With this feature, users can now navigate between key resources more efficiently and troubleshoot issues faster.

Datadog has developed Transaction Queries to simplify the process of aggregating logs according to shared attributes and provide context for root cause analysis in complex distributed systems. These queries enable grouping log events from various services into broader insights, such as e-commerce data or web user activity, allowing users to answer key questions about processes or user journeys. By defining custom boundaries using start and end conditions based on query of log messages, Transaction Queries produce precise log groupings that are more meaningful and easier for stakeholders to understand. The feature provides efficient troubleshooting by automatically calculating performance indicators like duration and max severity, and surfacing additional detail such as the count of logs containing unique values or p95 of a useful metric.

Google introduced Core Web Vitals in May 2020 as a set of three metrics that assess a site's UX performance. These metrics focus on load performance, interactivity, and visual stability. In May 2021, Core Web Vitals will be incorporated into Google’s PageRank algorithm, making it crucial for organizations to monitor these metrics. The collection and analysis of Core Web Vitals are essential not only for frontend engineering teams' usability goals but also for an organization's bottom line. Core Web Vitals consist of three components: Largest Contentful Paint (LCP), First Input Delay (FID), and Cumulative Layout Shift (CLS). These metrics observe load performance, interactivity, and visual stability, respectively. By collecting Core Web Vitals within RUM, Datadog enables users to incorporate these metrics into the broader picture of their web app's user-facing performance. Datadog provides a comprehensive view of Core Web Vitals scores through its Performance Overview dashboard and allows users to troubleshoot suboptimal scores in the RUM Explorer. Additionally, Datadog Synthetic Monitoring enables proactive validation of Core Web Vitals scores before users encounter issues. By monitoring these metrics, organizations can maintain a seamless user experience and ensure their site ranks well on Google.

The Core Web Vitals are a set of metrics that represent the most important indicators of a site's UX performance, focusing on load performance, interactivity, and visual stability. These metrics simplify UX metric collection by signaling which frontend performance indicators matter the most. Google recommends that a Cumulative Layout Shift score should be less than 0.1 for optimal performance. Datadog enables teams to monitor Core Web Vitals from an out-of-the-box dashboard, troubleshoot suboptimal scores in the RUM Explorer, and use Synthetic browser tests to proactively monitor Core Web Vitals scores in any environment. By incorporating these metrics into the broader picture of web app user-facing performance, teams can characterize load performance, identify bottlenecks, and maintain a seamless user experience.

Red Hat Gluster Storage is a distributed file system designed for scalability and flexibility across various environments. It integrates with Datadog to provide comprehensive visibility into the health of clusters, nodes, volumes, and bricks. The integration enables monitoring, alerting, and correlation of data from the Gluster file system alongside other telemetry sources in an organization's stack. Key metrics such as disk space usage, brick size, and cluster health can be monitored to ensure optimal performance and availability. Datadog also allows setting forecast alerts for early detection of potential space issues. With this integration, users can effectively manage their Red Hat Gluster Storage deployments and maintain the overall health of their distributed file systems.

Datadog has introduced Template Variable Associated Values, which dynamically presents the most relevant values for template variables to speed up troubleshooting. This feature helps users filter data and focus their exploration by placing the most relevant values within easy reach. As a user selects a value for each template variable, Datadog automatically finds the associated values for the dashboard's other template variables and places them at the top of the list. The associated values make it easier to move from a broad context to a more granular view of a key subset of hosts and services. This feature does not require any configuration and can be used with over 650 technologies.

Datadog dashboards have been enhanced to provide more effective troubleshooting and monitoring of infrastructure and applications by introducing template variables, which allow users to focus on specific subsets of hosts, containers, or services based on tags or facets. Template variable associated values enable users to isolate the most useful data, making it easier to filter and explore their dashboard data. These new features can be used with no additional setup, allowing users to start using them right away to quickly zoom in on relevant data from more than 850 technologies. The updated feature set also enables users to apply custom tags to create dimensions that allow for exploration of data in ways that make sense for their business, and can be easily shared as saved views for troubleshooting across teams.

Datadog has integrated with Red Hat Gluster Storage, allowing users to gain comprehensive visibility into the health of their clusters and their constituent nodes, volumes, and bricks. The integration ingests key metrics from Red Hat Gluster Storage's gstatus command line tool, as well as cluster logs, providing a high-level overview of deployment health. With this integration, organizations can scale out smartly by monitoring available disk space, brick size, and error logs to ensure optimal performance and availability. Datadog also enables users to track the health and status of their bricks, set alerts for potential issues, and pivot to Logs Explorer to diagnose errors. The integration is now available in Agent 7.26, offering a comprehensive view of the entire stack in one place.

Datadog has announced that Timber Technologies, creators of Vector, is joining the company. This acquisition aims to enhance Datadog's monitoring tools by providing customers with more control over their observability data ingestion, enrichment, storage, and routing processes. Vector is a vendor-agnostic data platform designed to collect, transform, and route logs, metrics, and traces from on-premise and cloud environments. The integration of Timber Technologies' expertise will enable Datadog to create more efficient and cost-effective data pipelines for customers while addressing compliance concerns and avoiding vendor lock-in.

Renaud Boutet from Datadog announces the acquisition of Timber Technologies, the company behind Vector, to enhance monitoring tools with robust yet flexible solutions for customers' existing workflows. The addition of Vector will provide users with more control over how their observability data is ingested, enriched, stored, and routed, enabling cost-efficient data pipelines in both cloud and on-premise environments. Vector is a vendor-agnostic data platform that allows users to collect, enrich, and transform logs, metrics, and traces from various sources, and route them to desired destinations. The integration of Vector with Datadog will provide new integrations, sophisticated capabilities, and a seamless user experience, aiming to break down silos and provide flexible solutions for observability data management.

Maël Nison, Senior Software Engineer at Datadog, shares his journey from growing up on a remote island to becoming the principal maintainer of Yarn, an important open source tool in the JavaScript world. After moving to France and discovering programming through a school club, he developed a passion for creating software that streamlines workflows. He joined Facebook where he was identified as a potential contributor to the nascent Yarn project. Since then, Yarn has evolved into a community-driven open source project with Maël wearing many hats throughout his involvement. Despite leaving Facebook and joining Datadog, Maël continues to lead Yarn while working on new challenges. His story highlights the importance of communication, adaptability, and empathy in technology roles.

Daniel Maher from the Community team at Datadog sat down with Maël Nison, Senior Software Engineer on the Frontend Platform team, to talk about his journey from childhood on a remote island to becoming the principal maintainer of Yarn. Growing up in France and later studying engineering in Paris, Maël developed an interest in programming at a young age, initially using DarkBASIC to create games. He went on to study in Canada before joining various organizations, including start-ups, where he honed his skills and learned to work with different teams. Eventually, he joined Facebook's Yarn project, where he became the driving force behind its development and growth into a community-driven open source tool. After leaving Facebook, Maël joined Datadog, where he continues to work on improving developer experience and is involved in various open source projects, including Yarn and VSCode improvements. Throughout his career, Maël has emphasized the importance of communication with users and making informed trade-offs when implementing solutions.

Over the past two decades, the Zend Engine, which powers PHP, has evolved to accommodate specific improvements. However, the primary observability hook that tracers, profilers, and debuggers used to observe PHP’s function-call behavior has not evolved alongside the advancements made to the Zend Engine. This aging hook had an increasingly adverse effect on observed PHP application runtimes. The release of PHP 8 includes changes that bring modern observability to the PHP runtime. Our team, along with help from the PHP internals community, developed and shipped the new observer API. The observer API is designed to combat all of the negative side effects of function-call-interception hooks mentioned previously. It introduces a more generic concept to the engine: observer.

The Zend Engine's function-call-interception hooks have been a challenge for observability, leading to performance issues and crashes. The new observer API in PHP 8 addresses these concerns by providing a more efficient and reliable way to instrument functions and methods. It introduces opcode specialization, allowing extensions to target specific functions of interest without notifying every extension for every function call. The observer API also supports targeting userland functions and is compatible with the new just-in-time compiler (JIT) added in PHP 8. Additionally, it eliminates the need for forwarding along hooks to neighboring extensions, making it "noisy neighbor resistant." This allows multiple tracers, profilers, and debuggers to run alongside each other without crashes. The future of observability in PHP now has a single point of reference under the `ZEND_OBSERVER` name, enabling further advancements in this area.

Datadog is committed to open standards and supports AWS Distro for OpenTelemetry's public preview. This collaboration allows customers to send metrics and traces to any supported monitoring backend, including Datadog. The partnership helps users improve their monitoring workflows regardless of their architectures. AWS Distro extends the CNCF OpenTelemetry project by collecting metadata from AWS resources and trace data from AWS SDK and AWS X-Ray. It is carefully optimized, secured, and tested to ensure performance and stability. Users can configure AWS Distro for OpenTelemetry to send data to Datadog by adding a datadog exporter to their configuration YAML file along with their Datadog API key. This collaboration strengthens the open standards movement and enhances data portability for users.