December 2024 Summaries

New Relic AI is transforming engineering workflows by leveraging large language models (LLMs) to simplify complex tasks related to system data interaction, debugging, and observability. By allowing engineers to query system data in plain language, it streamlines processes such as debugging, writing NRQL queries, understanding dashboard insights, ensuring alert coverage, and setting up synthetic monitoring. This approach reduces the time and cognitive load associated with traditional methods of analyzing logs and creating custom queries, allowing engineers to focus on resolving critical issues and optimizing system performance. New Relic AI enhances efficiency by summarizing errors, generating queries, and providing actionable insights quickly, making it a valuable tool for building resilient, high-performing systems.

New Relic AI, as discussed by software engineer Mehreen Tahir, revolutionizes the way engineers interact with system data through its use of large language models (LLMs), enabling tasks like debugging, writing queries, understanding dashboards, and ensuring alert coverage to be performed more efficiently using plain language commands. This tool transforms traditional, time-consuming processes by allowing engineers to ask questions in natural language instead of manually analyzing logs or constructing complex queries. It simplifies synthetic monitoring, streamlines workflows, and helps engineers focus on high-priority tasks by providing immediate insights and automated solutions, which significantly reduces cognitive load during high-pressure situations. Overall, New Relic AI is presented as a game-changing tool for observability, designed to make engineering work easier, faster, and less stressful.

OpenTelemetry (OTel) provides a standardized framework for collecting and exporting telemetry data, which is crucial for modern application monitoring. When integrated with New Relic, a comprehensive observability platform, users can leverage New Relic Query Language (NRQL) to extract actionable insights from OpenTelemetry data. The blog post details the process of storing and querying this data in the New Relic database (NRDB), emphasizing the importance of understanding different OpenTelemetry data types—such as counters, up-down counters, histograms, and observable counters—and their mappings to New Relic's data storage format. By following a step-by-step guide on instrumentation, data storage, and querying with NRQL, users can enhance their application's observability, optimize system performance, and gain valuable insights into metric trends and performance over time. The authors, Retish Perum and Lavanya Chockalingam, share their expertise and emphasize that the solutions presented are environment-specific and not part of New Relic's commercial offerings.

Service levels, including Service Level Agreements (SLAs), Service Level Objectives (SLOs), and Service Level Indicators (SLIs), are essential frameworks that help ensure service reliability and customer satisfaction. SLAs are legal contracts between service providers and customers, detailing the expected quality of service and the consequences if those expectations are not met. SLOs define the specific goals for system availability and performance, while SLIs are the metrics used to measure these goals. Site Reliability Engineering (SRE) practices emphasize the importance of these frameworks, and tools like New Relic assist in monitoring and adjusting these metrics to align with evolving user expectations and technological capabilities. Effective service level management, which involves setting and adjusting SLOs and SLIs, helps organizations maintain high service quality and comply with SLAs, thereby enhancing customer trust and operational efficiency.

OpenTelemetry (OTel) provides a standardized method for collecting and exporting telemetry data, crucial for modern application monitoring. When integrated with New Relic's Intelligent Observability Platform, users can maximize insights from their OpenTelemetry data. The blog post explains how to query OpenTelemetry data in New Relic using the New Relic Query Language (NRQL), detailing how OpenTelemetry data is stored in New Relic's database (NRDB) and the process of utilizing various OpenTelemetry data types, such as counters, up-down counters, histograms, and observable counters. It covers the steps to instrument, store, and query these data types through Python scripting and emphasizes the importance of understanding OpenTelemetry's metric data models. Furthermore, the guide highlights the flexibility and precision of NRQL in querying data for analysis and optimization, offering detailed instructions on setting up and executing queries to gain valuable insights into application performance. By leveraging these tools, users can significantly enhance their application's observability and performance monitoring capabilities.

Application performance is crucial for user satisfaction and business success, as slow applications can lead to user frustration and financial losses. Effective tracing and logging are essential tools for optimizing performance by providing insights into application behavior, identifying bottlenecks, and diagnosing issues. Tracing allows developers to follow requests through application components to pinpoint delays, while logging records important events and errors for context. Together, these techniques offer a comprehensive view of application performance, enabling informed decision-making for optimization. Tools like OpenTelemetry and New Relic support distributed tracing and centralized logging, making it easier to track requests across multiple services and analyze data effectively. Automating performance monitoring with real-time alerts ensures proactive issue resolution, while integrating monitoring into CI/CD pipelines helps identify performance issues early in development. Advanced techniques like machine learning further enhance optimization by predicting potential issues. Adhering to best practices ensures effective tracing and logging, avoiding pitfalls like over-logging and ensuring real-time monitoring for prompt issue detection and resolution.

Modern retail experiences are increasingly complex, spanning apps, stores, and distribution centers, requiring retailers to provide seamless, personalized digital experiences to maintain customer engagement and brand loyalty. The New Relic Retail Solution addresses these challenges by combining Intelligent Observability with AI-driven Digital Experience Monitoring (DEM) to offer a comprehensive view of the retail journey, from online to physical interactions. This integration allows retailers to proactively prevent disruptions, enhance operational efficiency, and ensure compliance with standards like PCI DSS. By leveraging a unified telemetry data cloud, New Relic provides real-time insights through application performance monitoring, infrastructure monitoring, and log management, enabling retailers to optimize performance and improve customer satisfaction. New Relic's platform supports open standards and integrates with various tools, facilitating dynamic monitoring and vendor neutrality, which helps retail businesses to enhance observability and drive growth by delivering superior end-user experiences.

Telemetry refers to the automated remote collection and analysis of data through wired or wireless communications, playing a crucial role in various industries such as IT, healthcare, aerospace, and more. It facilitates observability by enabling instruments to measure characteristics and produce essential data, which helps organizations make insight-driven improvements. Telemetry has transformed traditional data collection methods, such as nurses manually recording patient vitals or meteorologists predicting weather by observing the sky, by incorporating remote monitoring systems and satellite data. In IT, telemetry is integral to the observability process, allowing for the collection and analysis of environmental, performance, and operational data, which is used by developers, engineers, and business managers to optimize systems and develop business strategies. Despite its benefits, telemetry faces challenges such as managing large volumes of data, ensuring data protection, maintaining interoperability, and handling costs associated with evolving technologies. Various industries leverage telemetry to enhance operations, from IT infrastructure optimization to healthcare quality maintenance and automotive advancements, demonstrating its critical role in today’s data-driven world.

Modern retail experiences are becoming increasingly complex, requiring seamless integration across apps, stores, and distribution centers to maintain customer satisfaction and brand loyalty. New Relic Retail Solution addresses these needs by combining Intelligent Observability with AI-driven digital experience monitoring, offering a comprehensive view of customer interactions from online platforms to physical locations. This platform enables retailers to proactively prevent disruptions, optimize performance, and enhance customer experiences, leading to significant improvements in downtime reduction, customer satisfaction, and retention. It also supports compliance with PCI DSS standards, offering tools for logging, monitoring, and vulnerability management. By adopting open standards like OpenTelemetry, New Relic facilitates vendor independence and cost reduction. Through its integrated approach, New Relic empowers retailers to make real-time, data-driven decisions, ensuring operational efficiency and driving business growth.

Telemetry involves the automated remote collection and analysis of data through various communication methods, playing a crucial role in industries such as IT, healthcare, aerospace, and more. It enables observability by providing data that is used to optimize systems and improve decision-making processes. Historically, data collection was manual, but telemetry now allows for automated data gathering, transmission, and storage, facilitating insights via machine learning and data analytics. The data can include environmental, performance, and operational metrics, which are critical for monitoring and troubleshooting. Despite its benefits, telemetry faces challenges such as data overload, latency, security concerns, interoperability, and cost. Industries leverage telemetry to enhance infrastructure, healthcare, automotive technologies, weather forecasting, astrophysics, and defense operations. Telemetry's integration into observability systems helps organizations optimize performance and develop new business opportunities.

Application performance is crucial for user satisfaction and business success, with slow or unresponsive applications potentially leading to user frustration and financial losses. To optimize performance, tracing and logging are essential techniques that offer insights into application behavior, helping identify bottlenecks and diagnose issues. Tracing tracks the flow of requests to pinpoint delays, while logging records important events and errors to provide context for analysis. Together, these methods enable informed decision-making and optimization efforts. Tools like OpenTelemetry and New Relic facilitate tracing and logging by supporting distributed tracing and centralized logging, respectively. Automating performance monitoring and incorporating advanced techniques like machine learning into CI/CD pipelines can further enhance application performance, allowing developers to proactively address issues and ensure a seamless user experience. Following best practices, such as choosing the right tools and maintaining data consistency, is vital for effective performance optimization.

Real user monitoring (RUM) is a technique that provides insights into user experiences by capturing real-time, client-side data from users interacting with web properties. Unlike traditional performance monitoring, which focuses on server-side metrics, RUM offers a user-centric perspective by analyzing browser events such as page load times, errors, and user interactions. It is a vital component of digital experience monitoring (DEM) and helps businesses optimize user satisfaction, resolve issues swiftly, enhance conversion rates, and make data-driven decisions. The process involves data capture through embedded JavaScript tags, data processing for actionable insights, and data visualization for quick interpretation and problem-solving. RUM is beneficial across various industries, from ecommerce to mobile apps, by improving user experiences and optimizing website performance. While RUM provides comprehensive real-time insights, it poses challenges like generating baselines, handling vast data, and limited visibility into user context. New Relic's RUM solution offers a platform for capturing and analyzing these metrics, aiding in proactive performance optimization and seamless user journeys across digital platforms.

Observability is a practice that enhances the understanding of a system’s performance by analyzing the data it generates, surpassing traditional monitoring by providing deeper insights into system behavior. It involves the use of observability platforms to collect, store, analyze, and visualize data, including metrics, events, logs, and traces (MELT), allowing engineers to make informed decisions that enhance system reliability and performance. Observability tools, which integrate open instrumentation and AI operations, empower teams to proactively resolve issues, optimize systems, and improve customer experiences while fostering innovation. The importance of observability in modern digital businesses is underscored by its ability to provide a real-time, interconnected view of operational data, making it crucial for managing the complexities of cloud-native microservices and distributed systems. According to the 2024 Observability Forecast, a significant number of organizations have reported improved system reliability and a substantial return on investment from their observability initiatives, highlighting its value in enhancing team efficiency and driving business growth.

Auto-remediation is highlighted as a significant advancement in observability, minimizing human intervention and reducing the mean time to resolve issues. This guide details the process of implementing auto-remediation using New Relic and AWS EventBridge, specifically addressing high memory utilization in Amazon EC2 instances by automating their restart. The EC2 instance's metrics are monitored by New Relic, and when a threshold is exceeded, an alert triggers a notification to EventBridge, which executes a rule to restart the instance. The guide emphasizes configuring the integration between New Relic and EventBridge, including dynamically supplying the EC2 instance ID through alert notifications and setting up AWS roles and policies to automate the remediation process. The document also provides instructions on creating rules in EventBridge and configuring automation parameters to ensure the correct EC2 instance is targeted. Testing and validation procedures are included to confirm the successful execution of the AWS-RestartEC2Instance runbook, with additional options for creating dead letter queues or logging events in CloudWatch for troubleshooting purposes.

Auto-remediation, which aims to reduce human intervention and minimize mean time to resolve (MTTR), is effectively implemented through the integration of New Relic and AWS EventBridge, as illustrated in a comprehensive guide using an Amazon EC2 instance with high memory utilization. The process involves monitoring the EC2 instance with New Relic's infrastructure agent to trigger an alert when memory usage surpasses a predefined threshold. This alert sends a notification to EventBridge, which then executes a rule to restart the instance automatically. The guide emphasizes configuring the New Relic to EventBridge integration, including dynamically supplying the EC2 instance ID from alert notifications to ensure accurate remediation, and creating an SSM Automation Role in AWS IAM for rebooting EC2 instances. It also details setting up EventBridge rules to parse notifications from New Relic and restart the correct instance, highlighting the importance of specifying automation parameters and execution roles. Testing and validation steps are included to ensure the auto-remediation process functions correctly, with suggestions for troubleshooting and additional integration options provided for users interested in leveraging New Relic's capabilities further.

The Payment Card Industry Data Security Standard (PCI DSS) establishes security requirements to protect cardholder data environments, and New Relic has achieved PCI DSS Level 1 certification, allowing its customers to meet monitoring needs across both PCI-regulated and non-regulated environments. New Relic ensures compliance with PCI DSS version 4.0 by offering customizable logging retention and monitoring features, automated alerts, and centralized vulnerability reporting through its Security RX, which helps streamline vulnerability management across software and infrastructure. Additionally, New Relic allows for consolidation of infrastructure data, enabling enriched monitoring and insights on business risk. Customers with PCI DSS obligations can use New Relic to fulfill their compliance requirements and obtain an Attestation of Compliance.

The Payment Card Industry Data Security Standard (PCI DSS) is a set of security requirements to protect cardholder data environments, and New Relic has achieved PCI DSS Level 1 certification, enabling it to support both PCI-regulated and non-regulated environments. New Relic offers solutions such as customizable logging retention and monitoring, centralized vulnerability reporting, and infrastructure consolidation to help customers meet PCI DSS requirements. Specifically, Requirement 10 of PCI DSS version 4.0 focuses on log retention and monitoring, Requirement 11 on security testing practices, and Requirement 12 on validating PCI DSS scope; New Relic addresses these through features like Live Archives, Security RX, and integrated infrastructure monitoring. The blog encourages customers with PCI DSS obligations to leverage New Relic's platform for compliance and provides information on obtaining the company's Attestation of Compliance. The views expressed are those of the author, Casey Petry, and readers are invited to discuss further on New Relic's Explorers Hub.

At the recent AWS re:Invent, New Relic announced new product integrations with AWS, including a collaboration between New Relic AI and Amazon Q Business, aimed at enhancing enterprise productivity by integrating real-time data insights directly into business workflows. This initiative is part of their mission to eliminate digital interruptions through their Intelligent Observability Platform, which is now connected with Amazon Q Business to provide deep insights and intelligent recommendations, thereby streamlining complex workflows and automating incident responses. The collaboration allows New Relic AI to merge with Amazon Q Business's capabilities, facilitating faster and more informed business operations by breaking down team silos and centralizing critical observability insights. This integration offers a one-step observability solution for AWS, enabling enterprises to manage AWS services more efficiently through automatic instrumentation and AI-powered dashboards, eliminating the need for manual tagging and reducing operational overhead. By leveraging natural language and AI-enhanced insights, the connected solution empowers users across various roles to resolve issues more effectively, democratizing access to observability insights and enhancing overall enterprise productivity.

The collaboration between New Relic and AWS announced at AWS re:Invent strengthens their integration by connecting New Relic AI with Amazon Q Business, creating a unified platform for intelligent observability and enhanced workflow efficiency. This partnership leverages New Relic's Intelligent Observability Platform to provide deep insights and recommendations by combining Amazon Q Business's retrieval augmented generation capabilities with real-time monitoring. The integration allows users to detect and resolve operational issues, such as application performance slowdowns and network bottlenecks, directly within Amazon Q Business, improving decision-making and response times without the need to switch between tools. Additionally, New Relic's one-step observability for AWS simplifies AWS services onboarding and monitoring by automatically correlating performance data across applications and infrastructure, eliminating the need for manual tagging and enhancing enterprise productivity through AI-powered dashboards and anomaly detection. This collaboration aims to democratize observability, allowing users of varying expertise levels to access insights and streamline incident response by automating workflows and breaking down team silos.

In the realm of software development, the principle of "you build it, you own it" underscores the necessity for engineering teams to integrate security into their workflows, yet many organizations struggle to balance compliance demands with effective security practices. The article highlights the challenges of security theater, where compliance is often prioritized over genuine security enhancements, and introduces a solution through the integration of New Relic Security RX with FOSSA. This integration offers a comprehensive security approach by combining FOSSA's build-time software composition analysis with New Relic's runtime insights, providing real-time accuracy, streamlined remediation, actionable intelligence, and compliance assurance. Through this collaboration, the aim is to reduce legal risks, enhance security posture, and foster a culture of continuous improvement, making compliance and security more manageable and effective for developers.

In the evolving field of software development, the principle of "you build it, you own it" is becoming fundamental, emphasizing the need for engineering-centric security advancements such as integrated continuous testing and runtime insights. However, these innovations remain largely accessible to well-resourced organizations, leaving others reliant on anecdotal security measures. The integration of New Relic Security RX with FOSSA aims to bridge this gap by providing a comprehensive security solution that combines build-time software composition analysis with real-time production insights. This approach moves beyond mere compliance to focus on effective security practices, enabling developers to prioritize critical vulnerabilities and streamline remediation efforts. By delivering actionable intelligence and ensuring compliance, the integration fosters a culture of continuous improvement, positioning developers at the forefront of a secure and reliable digital ecosystem.