July 2021 Summaries

A company's Incident Intelligence product was taken offline due to an engineer mistakenly deleting a subscriber resource in Google Cloud Platform, which was used in production. The team took measures to prevent similar issues, including setting up a smoke test using synthetics and New Relic Alerts. The system, which processes incident data from various sources, was continuously tested to ensure that it was functioning correctly, with a heartbeat test regularly checking for data flow between input and output components. This added layer of protection provides an extra layer of reliability for the company's customers.

An unexpected outage in New Relic's Incident Intelligence product, caused by the accidental deletion of a critical Google Cloud Platform resource, led to the implementation of a robust smoke testing system to prevent future incidents. The system employs New Relic Alerts and synthetics to simulate incidents and ensure continuous data flow through the pipeline by periodically inserting and checking messages, with alerts triggered if expected signals are not received. This proactive measure provides an additional layer of protection for customers and enhances the reliability of New Relic's services, as explained by Shy Peleg, Director of Software Engineering at New Relic, who highlights the importance of detecting and addressing potential system failures swiftly.

The OpenTelemetry project offers three primary data sources: traces, metrics, and logs. Traces capture details of individual requests, providing detailed information about each request's latency and performance. The Trace API allows developers to create spans that can be associated with a globally unique identifier, enabling the propagation of trace context across service boundaries. The Metric API enables the collection of aggregated measurements, such as CPU utilization and request duration, while the Log API provides a structured text record in a time-stamped format for filtering by strategic attributes. These data sources are used to monitor and analyze application performance, providing valuable insights into system behavior and enabling data-driven decision-making.

OpenTelemetry is an open-source project that provides a framework for instrumenting, generating, and collecting telemetry data from software applications, focusing on three primary data sources: traces, metrics, and logs. Traces capture details of requests through a system using spans to represent operations, while metrics offer aggregated measurements such as CPU utilization and request duration. Logs provide time-stamped text records in a structured format. The article discusses how developers can use OpenTelemetry's APIs and SDKs to integrate these data sources into their applications, enabling enhanced observability and performance monitoring. Additionally, it highlights the use of the Trace and Metric APIs for adding metadata and configuring collection processes, as well as the strategies available for using log data with OpenTelemetry. The blog post also references upcoming content that will further explore how to instrument applications with OpenTelemetry and send data to backend platforms like New Relic.

New Relic for Startups` is a partnership between `AWS Activate`, `New Relic`, and other companies like `Zendesk` and `Carta`. It provides full access to the New Relic cloud-based observability platform, including monitoring, troubleshooting, and optimization of all AWS services, with generous credits and discounts. The program aims to help early-stage startups build and scale software faster and with confidence, from idea to IPO. With this partnership, startups can achieve six key things: start with access to the same dev tools used by world-class engineering organizations, avoid expensive monitoring tools, get instant monitoring for their entire AWS environment, easily connect with the tools they already love, establish DevOps and data-driven engineering practices early and seamlessly, and get started without a sales conversation. The program is available exclusively through the AWS Activate console.

The OpenTelemetry project is an open source initiative that provides a set of components for instrumentation and observability, including APIs, SDKs, tools, specifications, and semantic conventions. The API allows application developers to instrument their code to generate telemetry data, while the SDK provides an implementation of the API for popular programming languages. The OpenTelemetry Protocol (OTLP) defines the encoding of telemetry data and specifies how it is exchanged between client and server. Semantic conventions enable consistent operations across different platforms and languages. The OpenTelemetry Collector serves as a central repository to receive, process, and export telemetry data, supporting various formats and exporting data to backend observability tools. Exporters translate telemetry data to the required format for transmission to the destination system.

OpenTelemetry is an open-source project that provides a comprehensive framework for generating, collecting, and exporting telemetry data, including metrics, logs, and traces, using a set of APIs, SDKs, and tools. The project defines a centralized collector and exporters to transmit telemetry data to various backend platforms, enhancing visibility into system performance. Its architecture decouples the API from the SDK, allowing flexibility in implementation across different programming languages. OpenTelemetry Protocol (OTLP) standardizes the data exchange format, while semantic conventions ensure consistency in how operations are recorded across platforms. The OpenTelemetry Collector serves as a central hub for processing and exporting telemetry data, supporting various open formats, and can be deployed as a gateway or agent. Exporters translate and transmit data to desired backend observability systems, allowing flexibility in data handling without redeploying services.

The winners of New Relic's inaugural Data Nerd Awards were recognized for their innovative use of observability to drive customer experiences and simplify software development and performance monitoring. PETRONAS DAGANGAN BERHAD won the Data Creativity category with its Map of Malaysia app, which provides visibility into the health of all 616 PETRONAS stations across Malaysia. Tokopedia took the New Data Nerd award for its observability-first strategy to transform its platform into a "Super Ecosystem" with key partner integrations and business dashboards. Telstra received the New Tech Innovator Award for delivering real user performance monitoring across its Salesforce environments, while Talabat (Delivery Hero) won the DevOps Heroes Award for their commitment to DevOps best practices and full instrumentation with New Relic. These winners demonstrate how New Relic's customers are using observability to drive innovation and customer satisfaction.

New Relic's inaugural Data Nerd Award celebrates organizations worldwide that excel in observability and data innovation, with winners including PETRONAS DAGANGAN BERHAD, Tokopedia, Telstra, and Talabat. PETRONAS DAGANGAN BERHAD received the Data Creativity award for their innovative Map of Malaysia app, enhancing visibility and customer experience across 616 PETRONAS stations. Tokopedia was recognized as the New Data Nerd for their observability-first strategy, transforming their platform into a 'Super Ecosystem' and improving performance analysis. Telstra earned the New Tech Innovator award by implementing real user performance monitoring in their Salesforce environments, improving operational insights for 10,000 agents. Talabat won the DevOps Heroes award for integrating New Relic into their tech stack, optimizing resources, and enhancing customer lifecycle tracking. Each winner utilized New Relic's tools to significantly advance their technological capabilities and improve user experiences.

Pixie is a next-generation open source observability tool that provides unparalleled visibility into Kubernetes applications by automatically capturing application profiles, full-body requests, system metrics, and more data. It solves the problem of blind spots in Kubernetes environments with its auto-instrumentation technology, allowing developers to get started quickly without code changes or redeployments. Pixie's integration with New Relic One enables developers to debug faster with code-level insights, providing instant baseline visibility, flamegraphs, function-level granularity, and more, empowering them to identify performance bottlenecks and correlate application behavior with infrastructure-level behavior. With this integration, New Relic customers can start using Pixie today without additional cost, following a guided installation process or exploring the Auto-telemetry with Pixie documentation.

Pixie, now generally available as part of New Relic One, enhances Kubernetes monitoring by providing code-level insights and eliminating typical monitoring gaps. This open-source observability tool automatically captures detailed application profiles, system metrics, and Kubernetes-specific data, enabling developers to debug faster without code changes or redeployments. Pixie's integration within New Relic One offers instant visibility and uses eBPF for auto-instrumentation, allowing users to gain insights into application performance through features like flamegraphs and full-body request analysis. Designed specifically for developers, Pixie addresses limitations of traditional Kubernetes monitoring tools by offering actionable, developer-centric insights and a comprehensive view of application and infrastructure interactions. With the integration, New Relic customers can leverage Pixie's capabilities at no additional cost, enhancing their ability to swiftly diagnose and resolve performance bottlenecks.

OpenTelemetry is an open-source project that provides a unified standard for service instrumentation and performance measurement. It offers a single application programming interface (API), libraries, integrations, and a software development kit (SDK) for programming languages like Java, Go, and Python. By adopting OpenTelemetry, organizations can streamline their monitoring activities, reduce the need to manually create monitoring solutions, and ensure vendor neutrality. Telemetry data is collected from remote or inaccessible devices and transmitted to a central location for monitoring, analysis, and reporting, providing insights into performance, behavior, and usage patterns of systems and applications. OpenTelemetry enables distributed tracing, metrics collection, context propagation, and plugin ecosystems, making it a valuable tool for observability and performance enhancement. The project is vendor-neutral, community-driven, and standardized, allowing developers to collaborate and promote better coverage, flexibility, and ubiquity as engineers from all over the world contribute to the instrumentation. With OpenTelemetry, teams can optimize their team hours, reduce blind spots or data silos, and monitor emerging technologies easily. By combining with observability backends, OpenTelemetry provides full observability and monitoring in one tool, making it an essential component for modern software development and performance optimization.

OpenTelemetry is an open-source project sponsored by the Cloud Native Computing Foundation (CNCF) that provides a unified standard for service instrumentation, facilitating performance measurement through a single API, libraries, and SDKs for languages like Java, Go, and Python. It replaces OpenTracing and OpenCensus to standardize telemetry data collection and transmission to backend platforms, offering features such as distributed tracing, metrics collection, and context propagation. This vendor-neutral approach enables interoperability and flexibility, mitigating the need for proprietary solutions while fostering community-driven innovation with contributions from over 300 companies. OpenTelemetry aims to enhance observability by providing rich insights into system behavior, essential for performance monitoring, issue detection, capacity planning, security monitoring, and compliance. The framework's interoperability and vendor neutrality allow development teams to switch backend observability tools without altering instrumentation, making it a future-proof choice for monitoring modern distributed systems. New Relic, a significant contributor to OpenTelemetry, supports this standard with native OpenTelemetry protocol and offers resources like tutorials and masterclasses to help users harness its capabilities for system optimization and troubleshooting.

New Relic One provides an Alert Condition Recommendation service that uses AI and machine learning to recommend specific metrics and signals to monitor for entities in a production application. This service helps users figure out what to monitor and set up thresholds that are sensitive enough to catch anomalies but not so sensitive that they trigger distracting false alarms. The recommendations can be easily added to APM entities without alert coverage, and the model uses tags associated with an entity to generate recommendations based on the quality of the tags. The service also provides dynamic baseline alerts with recommended threshold values, allowing users to choose between default or custom thresholds. By increasing alert coverage, New Relic One helps users maintain a high level of system health visibility.

Just as smoke detectors are essential for alerting homeowners to fires, production applications require alerts to signal when issues arise, though setting precise alert conditions can be challenging. New Relic One offers an Alert Condition Recommendation service that employs AI and machine learning to suggest specific metrics and signals for monitoring, allowing users to customize alerts according to their needs. The service leverages entity tags to enhance the precision of recommendations, dividing entities into clusters based on tag similarity and identifying common conditions for each cluster. In cases where data is insufficient, the model refers to a community-curated golden signals data set. The system also suggests appropriate thresholds for alerts by analyzing historical data, aiming to balance between over-alerting and missing critical issues. This service is designed to improve alert coverage efficiently, ensuring comprehensive system monitoring akin to having functional smoke detectors at home.

New Relic has partnered with Trend Micro to help customers further strengthen their cloud observability and security posture by integrating Conformity into New Relic's platform. This integration brings in all cloud infrastructure configuration data from Trend Micro Cloud One - Conformity, providing a complete cloud visibility strategy by complementing New Relic-powered observability with added visibility into cloud security and compliance posture. The integration uses Amazon Simple Notification Service (SNS) communication channel to enable Conformity data to flow into New Relic, as well as the Conformity Public API to fetch all Conformity checks every 30 days. This allows customers to create custom dashboards in New Relic to explore and track their Conformity data, including a CISO dashboard that tracks top Conformity rules across all AWS accounts. The integration also enables real-time threat monitoring events from Conformity integrated with New Relic's event API, and proactive anomaly detection using New Relic Lookout.

New Relic has partnered with Trend Micro to enhance cloud observability and security for AWS, Azure, and Google Cloud users by integrating Trend Micro Cloud One - Conformity into its platform. This integration allows users to automatically import cloud infrastructure configuration data into New Relic, thereby combining cloud visibility with security and compliance insights. Trend Micro Cloud One offers a comprehensive suite of security services addressing various aspects such as workload, network, and application security, while Conformity continuously evaluates cloud setups against industry standards to provide actionable intelligence for compliance and security posture improvements. The integration, facilitated by AWS solutions and architecture, allows real-time monitoring and proactive anomaly detection via New Relic's Applied Intelligence. This setup enables teams to filter and visualize data, ensuring timely identification and resolution of potential issues, and facilitates the creation of customized dashboards to monitor compliance and security events, helping organizations streamline their cloud operations and avoid non-compliance pitfalls.

New Relic is expanding its capabilities for monitoring Amazon Web Services (AWS), particularly for Amazon Elastic Kubernetes Services (EKS) containers deployed using AWS Fargate as a serverless deployment platform. This integration enhances observability of the Kubernetes cluster by injecting sidecar containers to collect metrics from kube-state-metrics, kubelet, and cAdvisor. The EKS Fargate integration supports both hybrid clusters with Fargate nodes and EC2 nodes, allowing for full visibility into the Kubernetes cluster. Additionally, it is compatible with New Relic's Kubernetes cluster explorer, enabling users to filter Fargate nodes by ComputeType or FargateProfile tags. The integration also improves dashboards, listing Fargate nodes and distinguishing them from standard nodes.

New Relic has expanded its monitoring capabilities to include Amazon Elastic Kubernetes Services (EKS) containers, specifically when deployed using AWS Fargate. This new integration allows for comprehensive monitoring by injecting a sidecar container into the Fargate node, enabling New Relic to collect data from kube-state-metrics, kubelet, and cAdvisor for full observability of the Kubernetes cluster. The integration supports any Fargate setup, whether the cluster consists solely of Fargate nodes or a mix with EC2 nodes, with compatibility for the Kubernetes cluster explorer in New Relic One, providing a detailed view of the cluster. Improved Kubernetes dashboards now distinguish between standard and Fargate serverless nodes, allowing users to filter Fargate nodes using specific tags. This development aims to leverage the benefits of serverless computing, such as reduced costs, diminished server maintenance, and increased productivity, by enabling developers to focus on software creation rather than infrastructure management.

Google Lighthouse is a free and open-source tool that provides a snapshot of your website's performance and search engine optimization (SEO). It can be integrated into your New Relic One dashboard to track metrics continuously and receive alerts if performance metrics worsen. Using Google Lighthouse, you can analyze site performance, accessibility, and SEO, and gain insights into how quickly your site loads and how long it takes for users to see content. By adding Lighthouse data to your New Relic One dashboard, you can get baseline metrics that aren't affected by real-world conditions, making your dashboard more versatile and leading to new insights into your site's performance. You can also set up alerts on incoming data from Google Lighthouse to monitor performance and receive notifications when issues arise. Integrating Lighthouse with New Relic One allows for easy visualization of metrics and enables you to focus on important metrics like accessibility and SEO, while gaining a better understanding of how your site is performing.

Google Lighthouse is a free, open-source tool used for assessing website performance and SEO, but it only provides a snapshot of performance at a given moment. Integrating Lighthouse with New Relic One allows for continuous tracking of performance metrics alongside other data, providing more comprehensive monitoring and the ability to set alerts for performance drops. The integration process involves using the Google PageSpeed Insights API and setting up a monitor in New Relic to regularly gather Lighthouse data in JSON format. By adding this data to a New Relic dashboard, users can visualize metrics like First Contentful Paint and Largest Contentful Paint and set alerts to quickly identify and address performance issues. This integration helps in gaining better observability and monitoring maturity, essential for developing robust frameworks.

AWS Control Tower is a service that helps customers set up and govern their multi-account AWS environment, providing automated landing zones with best-practice blueprints for identity, federated access, centralized logging, and account structure. The service offers instant observability through the New Relic One platform, which provides continuous visibility into AWS environments, including accounts provisioned, guardrails enabled, and compliance status against established policies. The New Relic Quick Start solution streamlines the observability of landing zones, allowing customers to deploy the integration in minutes using AWS CloudFormation, granting instant observability, frictionless onboarding, effortless management, and flexible monitoring capabilities.

AWS Control Tower offers a streamlined approach to setting up and managing multi-account AWS environments by providing best-practice blueprints for identity, access, logging, and account structures, ensuring compliance and operational agility at scale. The integration with New Relic, an AWS Partner Network, enhances observability by automatically making accounts observable upon launch or enrollment, facilitating centralized governance and operational oversight across multiple accounts and regions. The New Relic AWS Quick Start solution simplifies deployment through one-click setups and allows users to manage operational data from a single platform, helping with governance, risk management, and financial management through consolidated dashboards and anomaly detection tools. This integration supports compliance and auditing by utilizing data from various AWS services to build centralized dashboards, offering a comprehensive view of costs and budgeting for better financial decision-making. Users can also leverage New Relic's Lookout feature for detecting anomalies across AWS workloads, aiding in operational excellence by unifying operations tooling and insights into one cohesive framework.

The author of this text is a software developer who initially struggled with understanding observability but eventually learned about OpenTelemetry and its benefits for modern apps. They created a beginner-friendly resource to instrument their own apps using OpenTelemetry, which includes tutorials and code snippets. The author explains how distributed tracing shows the path a request takes through an application, helping developers identify performance issues. They demonstrate how to spin up a simple movie app with two microservices, use Docker to install Zipkin for tracing, and export data to New Relic using the OpenTelemetry Collector. The author provides a step-by-step guide on how to instrument an app with OpenTelemetry, including setting up a trace provider, adding a span processor, using Docker to spin up Zipkin, and exporting data to New Relic.

Observability, particularly through OpenTelemetry, is crucial for maintaining modern applications' uptime and performance, especially as systems grow more complex. Initially unfamiliar with its importance due to a frontend background, the author learned about OpenTelemetry, a tool for app and website monitoring, and collaborated with freeCodeCamp.org to create a beginner-friendly video course. This course covers microservices, tracing, and observability. The article guides readers through using OpenTelemetry to instrument a Node.js application, explaining the setup process with tools like Zipkin for distributed tracing, which helps visualize and analyze request paths through applications. It emphasizes the importance of visibility into system operations to identify performance bottlenecks and improve reliability, and discusses exporting telemetry data to platforms like New Relic for further analysis. The author encourages developers to explore OpenTelemetry and offers resources for implementing it, aiming to make tech accessible and foster inclusive education.

Smoke testing in production is a crucial aspect of ensuring seamless deployments and reducing risks for end-users. By leveraging synthetic monitors for automation, teams can ensure that critical functionalities are working as expected, catch major issues before they affect users, and improve overall software quality. Automated smoke tests can be integrated into continuous integration pipelines for even quicker results, while hybrid smoke testing combines manual and automated approaches to cover all the basics. To set up effective smoke testing in production, developers should define the scope, prepare the test environment, execute smoke tests, document results, investigate failures, take corrective actions, and continuously improve their process. Additionally, synthetic monitors like New Relic Synthetics can be used to automate smoke testing, providing performance, functionality, and availability testing for reliable continuous delivery. By incorporating smoke testing into their deployment processes, organizations can deliver reliable and high-performing software to their customers while reducing risks and downtime.

New Relic One offers a single UI for viewing data, making troubleshooting easier for Spring Boot developers with Microsoft Azure's latest integration. With this integration, developers can send their application data directly to New Relic One and take advantage of the powerful observability platform. The integration deploys the New Relic Java agent into a Spring Boot application, allowing users to visualize and troubleshoot their entire software stack within one connected experience. This unified view provides shared context across teams, making it easier to identify issues and optimize applications. Developers can monitor key aspects of their applications, including web transaction time, throughput, and error rate, as well as access features like New Relic Lookout without additional configuration. The integration is a simple three-step process using the command-line interface, and users can explore the platform after deploying their application and configuring the New Relic Java agent.

Smoke testing in production is a critical component of modern continuous delivery and deployment processes, ensuring that core functionalities of software applications work as expected with minimal configuration. This approach involves using synthetic monitors to automate tests for performance, functionality, and availability, thereby reducing risks for end-users and improving software reliability. Automated smoke tests allow for rapid feedback and quicker issue resolution by running in parallel with other tests, thus supporting continuous integration pipelines. The article highlights the importance of smoke testing both in production and staging environments to catch high-priority issues before they affect users and emphasizes the utility of synthetic monitors in tracking application performance and availability. Through custom dashboards and alert notifications, teams gain better visibility into deployments, enabling them to balance deployment velocity with service stability.

Spring Boot developers can now enhance their application monitoring and troubleshooting capabilities through the integration of Microsoft Azure with New Relic One, a powerful observability platform. This integration allows developers to send application data directly to New Relic One, enabling a unified view of their software stack for easier error detection and performance analysis. Azure Spring Cloud, which manages infrastructure for Java Spring Boot and .NET Core applications, simplifies deployment processes and automatically installs New Relic's Java agent, facilitating seamless visualization and monitoring of key performance metrics such as web transaction time, throughput, and error rates. The integration supports various data analysis features, including distributed tracing and custom dashboards, to help developers pinpoint and resolve issues efficiently. This collaboration between Azure and New Relic provides a comprehensive toolset for ensuring application reliability and performance, allowing teams to maintain fast, reliable, and available software systems.