November 2022 Summaries

Terraform is a powerful observability-as-code tool that enables users to define and manage cloud and on-prem resources through configuration files written in HCL (HashiCorp Configuration Language). However, working with HCL syntax can be challenging, especially when updating and adding new widgets to an existing New Relic dashboard. To overcome this, users can utilize JSON templates to make it easier to manage Terraform-provisioned New Relic dashboards without having to convert them to HCL syntax. With JSON templates, users can edit the dashboard in the New Relic UI, export the configuration into their Terraform project, and use extended features of New Relic dashboards, including series coloring and y-axis scaling. This allows users to create a New Relic dashboard in Terraform using JSON templates, which is part one of a three-part series on dynamically creating New Relic dashboards with Terraform. The prerequisites for this tutorial include provider version 3.4.0 or above, as well as having Terraform installed. Users can follow along with the sample code and instructions to create a dashboard using JSON, update an existing dashboard, and redeploy with minor tweaks. The next steps involve reading part two of the series on dynamically creating New Relic dashboards with Terraform and part three on generating dashboards from NRQL queries.

Terraform offers an observability-as-code tool that manages cloud and on-premises resources through configuration files using HCL but can be challenging to use with New Relic dashboards. To simplify this, JSON templates can be utilized, allowing for easier management without converting configurations to HCL syntax. Users can edit dashboards in the New Relic UI, export the configuration as JSON, and integrate it into their Terraform projects. This approach supports extended dashboard features like series coloring and y-axis scaling, enhancing customization capabilities. The blog explains how to implement this process, noting that Terraform provider version 3.4.0 or higher is required. Additionally, the series explores advanced dashboard creation methods, including dynamic generation using JSON with template variables and NRQL queries.

Sustainable IT aims to minimize environmental and climate impacts through infrastructure, coding, and operational techniques. Monitoring and measurement systems identify what's working and what isn't. The State of Green Cloud Software Practices outlines four main approaches: Operational efficiency focuses on reducing machines and resources required for applications and processes, which can cut down on carbon usage by up to five to tenfold. Architecting for minimal carbon involves designing software from an energy efficiency standpoint, while energy efficiency aims to reduce the amount of energy required to run applications. Hardware efficiency extends the longevity of hardware to conserve resources and limit device creation. Measuring sustainable IT requires creating key performance indicators (KPIs) based on technical and business metrics, such as proxy metrics for compute, network, and storage resources and business metrics like number of active users, completed transactions, and cost of cloud spend. By tracking these KPIs, organizations can quantify performance over time and reduce their energy usage to align with environmental and social values.

### Elasticsearch Monitoring with New Relic Ruby Agent The New Relic Ruby agent provides automatic instrumentation of versions 7.x and 8.x of the Elasticsearch gem, allowing for better monitoring of performance issues and errors related to requests to your Elasticsearch clusters. The agent uses metaprogramming techniques to insert instrumentation code into commonly used public methods of libraries, including the Elasticsearch gem. This allows for greater flexibility in configuring instrumentation options, such as capturing or obfuscating queries. The Ruby agent supports versions 7.x and 8.x for the Elasticsearch gem and provides additional features like capture_queries and obfuscate_queries configuration options to protect sensitive information. Upgrading to version 8.12.0 of the New Relic Ruby agent is required to monitor requests from your Elasticsearch client.

Elasticsearch, a widely-used open-source search and analytics engine, is integral in delivering speedy and relevant search results across various platforms like streaming services and grocery delivery apps. The New Relic Ruby agent, specifically from version 8.12.0, offers enhanced monitoring capabilities for Elasticsearch, particularly targeting versions 7.x and 8.x of the Elasticsearch gem, by employing metaprogramming techniques to automatically instrument key methods. This setup provides valuable insights into the performance and errors of Elasticsearch clusters within Ruby applications, with options to capture or obfuscate query details to maintain data privacy. The New Relic Ruby agent uses techniques like Module#prepend and alias_method chaining for instrumentation, addressing conflicts with SystemStackError troubleshooting advice for efficient monitoring. Additionally, the agent supports configuration options to either capture or obfuscate Elasticsearch queries, ensuring sensitive information is secure while providing detailed operation insights through caller_locations. Users are encouraged to upgrade their Ruby agent to leverage these capabilities and can contribute to the open-source project via GitHub while enjoying New Relic's free monitoring plan offering.

Sustainable IT aims to reduce environmental and climate impacts by employing various infrastructure, coding, and operational techniques, with a focus on energy efficiency and resource optimization. The approach includes operational efficiency, architecting software for minimal carbon emissions, enhancing energy efficiency, and maximizing hardware longevity, all of which contribute to reducing carbon footprints. Techniques like frugal data storage, green coding, and API call optimization are emphasized, while monitoring and measurement systems play a crucial role in identifying effective practices. Sustainable IT practices also incorporate key performance indicators (KPIs) that combine proxy and business metrics to evaluate energy usage against business outcomes, such as transaction completions and user activity, ensuring that sustainability efforts do not compromise business goals. This approach not only supports environmental objectives but can also result in cost savings, aligning with broader business sustainability strategies.

New Relic dashboards allow for flexible data visualization across various sources, utilizing pre-made templates, native platform visualizations, custom queries, and more. The latest enhancements include color customization options to highlight key information or match branding colors, null value settings to handle gaps in data, unit customization to improve clarity on charts, and a Markdown widget to add context and flair to dashboards. These new features enable users to create customized dashboards that cater to their specific needs, making it easier to analyze telemetry data across the entire stack.

Content Delivery Networks (CDNs) enhance the speed and efficiency of loading internet content through globally distributed servers, but they can be challenging to troubleshoot during high-demand periods, risking revenue and customer trust if not managed effectively. To address these challenges, New Relic offers integrations with leading CDN providers like Akamai, Amazon CloudFront, Cloudflare, and Fastly, enabling rapid monitoring of CDN performance through pre-built dashboards and alerts. These integrations allow users to track critical metrics such as latency, availability, and error rates, ensuring swift identification and resolution of issues. New Relic facilitates this by offering a free account with 100 GB/month of free data ingest and access to over 500 prebuilt quickstarts, supporting a wide range of telemetry data sources. The platform emphasizes ease of use and integration, providing detailed setup instructions and support documentation for each CDN provider, thereby empowering users to maintain optimal CDN performance and enhance customer experience.

New Relic dashboards provide a versatile platform for engineering teams to visualize and analyze telemetry data from various sources within a unified interface. These dashboards offer extensive customization options, such as adjusting chart colors for better data interpretation, managing null values to ensure clarity, and customizing units for enhanced chart detail. Users can also analyze different time ranges on the same dashboard and incorporate custom content using a Markdown editor. A sample dashboard JSON file is available for users to try out these features, and the blog provides guidance on setting up and customizing dashboards to optimize data visualization and analysis. The article emphasizes the flexibility and functionality of New Relic dashboards in improving observability and monitoring capabilities and encourages users to explore further customization options and resources.

Cloud-native architecture adoption is growing rapidly, with over 6.8 million cloud-native developers, but it also brings numerous challenges such as managing costs, dealing with short-lived components and variable microservices, lacking insight into underlying infrastructure, and increasing development and release cycles. These challenges can lead to inefficiencies, security risks, reliability issues, and difficulties in managing system configuration settings. To overcome these challenges, organizations need an observability solution that provides visibility across their cloud-native environment, including monitoring services, deployment markers, and vulnerability management.

During the third holiday season since the pandemic began, shoppers are still eager to buy gifts despite inflation concerns, with 56 percent planning to shop online according to a 2022 survey by the National Retail Federation and Prosper Insights & Analytics. Businesses aiming to maximize online and mobile sales during peak days like Black Friday and Cyber Monday should focus on optimizing uptime and performance well in advance, as outlined in a guide that uses New Relic's platform for actionable data insights. The guide emphasizes setting baselines and defining success, thorough advance planning, understanding the current environmental state, and classifying issues by business impact. It also advises setting up a command center, managing changes carefully, maintaining a big-picture focus, monitoring key customer journeys, optimizing payment processes, staying flexible, and preparing for future success. Emphasizing a year-round readiness approach, businesses are encouraged to use New Relic's tools and insights to navigate the holiday season effectively and maintain success throughout the year.

Cloud-native architecture is increasingly adopted by over 6.8 million developers, offering benefits alongside notable challenges such as cost inefficiencies due to utility pricing models, ephemeral microservices, and limited insight into underlying infrastructures. Security is a critical concern due to the lack of visibility, with the need for consistent policies across hybrid environments. Frequent deployment cycles necessitate coordination within CI/CD pipelines, while managing multiple systems and clouds demands comprehensive visibility. Configuration management can be problematic due to drift, and the vast flexibility can lead to analysis paralysis. Reliability issues require careful architectural planning and observability, while organizations must adapt culturally and skill-wise to a DevOps and CI/CD-focused environment. New Relic provides observability tools to address these challenges, offering unified visibility and deep insights across cloud-native infrastructures.

Observability is emerging as a critical priority for IT decision-makers, with nearly three-quarters of respondents in a survey commissioned by New Relic and Enterprise Technology Research indicating that C-suite executives champion its implementation to enhance core business goals. The 2022 Observability Forecast report reveals that mature observability practices can improve revenue retention by providing deeper insights into customer behaviors and reducing high-impact outages. The report highlights the need for transparent pricing models in observability tools and notes that 72% of respondents plan to maintain or increase their observability budgets in the coming year. With the trend towards cloud-native architectures and technologies like AI and IoT, companies are expected to have robust observability practices by 2025, emphasizing the importance of full-stack observability for faster incident detection and resolution.

Dealer.com, a prominent platform facilitating over 43 million vehicle views monthly, relies heavily on site performance for enabling seamless customer experiences in car purchases. The company's complex software infrastructure, comprising multiple services, can be challenging, as any single service issue can affect overall response times and degrade user experience. To address these challenges, Dealer.com employs New Relic's distributed tracing, which is a feature included with their application performance monitoring (APM) services at no extra cost. This tool provides comprehensive visibility into service dependencies and individual request paths, helping software teams quickly identify and resolve performance issues within their distributed systems. By automatically instrumenting application code, New Relic simplifies the process of managing trace data, supports diverse language frameworks, and adheres to common tracing standards like OpenTelemetry. This allows DevOps teams to optimize code efficiently, ensuring a smoother experience for end-users and maintaining system reliability in rapidly evolving cloud and microservices-based environments.

Upgrade your New Relic agent to reduce security vulnerabilities and improve monitoring capabilities, including distributed tracing, logs in context, and access to production telemetry data in your IDE. Additionally, upgrading your agents provides access to technical support and bug fixes, as well as the latest features and enhancements. To ensure a smooth upgrade process, follow best practices such as regularly updating to the latest agent version, reviewing agent versions, keeping consistent agent versions across services, using automation for deployment and upgrades, and reading release notes. By doing so, you can minimize downtime, reduce security risks, and optimize your application performance monitoring.

New Relic's application performance monitoring (APM) utilizes agents to collect telemetry data, emphasizing the importance of keeping these agents updated to reduce security vulnerabilities, enhance monitoring capabilities, and ensure access to technical support and bug fixes. Up-to-date agents provide software patches that address security risks, introduce features like distributed tracing for better system visibility, and allow log context integration for faster troubleshooting. Regular updates also ensure compatibility with the latest software developments and maintain consistent troubleshooting experiences across services. Best practices for maintaining agent updates include regular version reviews, using automation for deployments, and staying informed through release notes. The blog post is contributed by Sonya Judd and Lavanya Chockalingam, highlighting their expertise and perspectives, while also directing readers to New Relic's resources for further guidance and community support.

Deploying the New Relic infrastructure monitoring agent on Amazon EC2 instances, particularly on Amazon Linux, can be a swift and straightforward process, allowing users to start receiving data within 60 seconds. The process involves launching an EC2 instance, configuring advanced details like User data with a script that includes the New Relic license key, and installing the agent using tools like AWS CloudFormation or automation tools. Users need to have both an AWS and a New Relic account to perform the setup, with the option of using Amazon Linux 2 for the demonstration. This method of deployment is highlighted as efficient, with the potential to launch and configure multiple instances in a matter of seconds, making it a practical choice for development, testing, or production environments seeking rapid infrastructure monitoring capabilities.

Amazon CloudWatch Metric Streams, when integrated with New Relic, provides an efficient solution for monitoring AWS cloud infrastructure by enabling real-time visibility into system performance and operational events. This integration leverages the AWS Well-Architected Framework to guide users in building reliable and cost-effective cloud workloads while ensuring operational excellence through telemetry monitoring. By utilizing Amazon CloudWatch Metric Streams, businesses can access up-to-date telemetry data without the need for continuous querying, allowing for quick detection and response to operational incidents. The process involves setting up a metric stream in AWS, configuring New Relic to receive and analyze this data, and establishing alert policies for immediate notification of performance issues. This integration simplifies data management by eliminating the need for custom integrations for each AWS service, thereby streamlining the observability of cloud operations.