January 2022 Summaries

At New Relic, their mission is to make observability a daily practice for millions of engineers. In the second half of 2021, they accelerated this mission by launching ten major product updates and countless other enhancements. These updates include new core user type, New Relic CodeStream, Instant Observability (I/O), errors inbox integrated with Slack and Jira, model performance monitoring, infrastructure monitoring, OpenTelemetry protocol support, TISAX and ISO 27001 compliance, HIPAA compliance, AIOps, and a redesigned Alerts & AI tab experience. These enhancements provide more visibility into code performance, enable collaboration and prioritization of work directly from the IDE, detect patterns and outliers in log data, build and run custom applications, onboard new developers with observability best practices, and offer a powerful observability solution for AWS workloads and infrastructure. With these updates, New Relic aims to help engineers troubleshoot, make data-driven engineering decisions, and embed observability throughout the entire development lifecycle.

New Relic has significantly advanced its mission to make observability a daily practice for engineers by launching numerous product updates and enhancements throughout the latter half of 2021. Key developments include the introduction of core user types designed to enhance code performance understanding and collaboration, the integration of New Relic CodeStream for seamless in-IDE code review and debugging, and the New Relic Instant Observability (I/O) platform offering over 400 quickstarts for easy monitoring setup. Additional updates encompass the Errors Inbox feature integrated with Slack and Jira, now available in the EU, and model performance monitoring for AI and ML teams to break down visibility silos. Infrastructure monitoring has been improved to help teams quickly identify and resolve issues, while native OpenTelemetry protocol support has been bolstered to enhance visibility across systems. New Relic also assures compliance with TISAX, ISO 27001, and HIPAA to protect sensitive data, particularly in healthcare. Finally, AIOps enhancements have been introduced to reduce alert noise and streamline incident resolution, solidifying New Relic’s comprehensive observability platform offerings.

The author of this post purchased an 8 kW Tesla solar system with a 27 kWh battery backup and wanted to monitor its performance using New Relic One. The system has a widely used API, but the author preferred to use New Relic's Metric API for data submission. They created a script to pull data from the gateway API, massage it into a suitable format, and send it to New Relic One. The author then built a custom dashboard using New Relic One's features, including real-time power generation, consumption data, and battery charge level. The dashboard also includes weather data from OpenWeather and has alert functionality for low battery reserves. The author envisions expanding the dashboard to monitor other IoT devices in their home network, leveraging New Relic One's capabilities for infrastructure monitoring.

In 2020, the author installed an 8 kW Tesla solar system with a 27 kWh battery backup and decided to monitor the system using New Relic One, an observability platform traditionally used for web applications. The system comprises solar panels, an inverter, a gateway, and batteries, which allow the house to use solar power, store excess energy, and potentially return power to the grid. By leveraging the unofficial API of the Tesla system, the author was able to pull real-time data and send it to New Relic One using the Metric API, thereby creating custom dashboards for monitoring energy usage and efficiency. The process involved aggregating data from various sources, including weather information, to provide a comprehensive overview of the solar system's performance. The author appreciated the ease of use and flexibility that New Relic One offered, enabling them to visualize data without the need for complex coding or local server management. The author envisions further integration of IoT devices into New Relic One for a holistic home monitoring system and encourages others to try New Relic for infrastructure monitoring, highlighting the benefits of its free account offerings.

Kubernetes autoscaling allows you to automatically adjust the number of running instances of an application based on observed resource utilization or custom metrics. There are two main types of autoscaling: Horizontal Pod Autoscaler (HPA) and Vertical Pod Autoscaler (VPA). HPA adjusts the number of pods, while VPA adjusts the CPU and memory resource requests of individual pods. By using New Relic's Kubernetes integration with Pixie, you can automatically scale your deployments based on metrics gathered from your applications and infrastructure services. This allows for improved availability, cost-effectiveness, and reduced manual intervention. To implement this solution, you need to deploy a minikube cluster, install the New Relic Metrics Adapter, test your NRQL query, configure your HPA, add load to trigger autoscaling, and follow best practices for Kubernetes autoscaling.

Managing resource allocation for applications on Kubernetes is complex, but the platform's autoscaling capabilities, particularly Horizontal Pod Autoscaling (HPA) and Vertical Pod Autoscaling (VPA), offer solutions for efficiently handling varying workloads. HPA adjusts the number of pods based on metrics like CPU usage, while VPA tunes the resource requests of individual pods dynamically. Using New Relic's integration, metrics can be harnessed to optimize this process, enabling automatic scaling of deployments without manual intervention and enhancing cost efficiency. The integration involves steps like setting up a minikube cluster, installing New Relic's Kubernetes components, and configuring the HPA to respond to external metrics, such as those provided by New Relic's NerdGraph API. Best practices for Kubernetes autoscaling include defining clear metrics, setting realistic resource limits, monitoring performance, and testing configurations to ensure effective and stable scaling. The ongoing refinement of autoscaling parameters, supported by tools like New Relic, is crucial for maintaining a resilient and cost-effective infrastructure.

Syslog forwarding allows for the correlation of Syslog messages with other observability data, enabling analysis and anomaly detection in network equipment and application layers. This feature supports FedRAMP customers by leveraging a single Docker image that collects SNMP and network flows, allowing for secure and compliant Syslog forwarding without increasing agent or platform management burden. The integration enables automatic detection of anomalies, surface patterns and potential issues with zero-configuration Log Patterns, and stays within FedRAMP compliance. With the release, customers can easily configure Syslogs with New Relic using a few simple steps, including logging into their account, preparing a Linux host, deploying a Docker container, configuring internal systems, and exploring Syslog streams in the New Relic platform.

New Relic has introduced secure Syslog forwarding, which is now available for FedRAMP customers, allowing users to send Syslogs to New Relic One with minimal configuration changes by using the same Docker image that collects SNMP and network flows. This integration aims to enhance network observability by correlating Syslog messages with other telemetry data, helping users analyze network performance, automatically detect anomalies with Lookout, and identify patterns without additional configuration. The solution supports FedRAMP compliance and offers a streamlined configuration process through New Relic's Instant Observability quickstart, which provides guided installation and an example dashboard. Users can manage their Syslog streams with New Relic's Logs UI, and FedRAMP customers can utilize guided installation for compliant endpoints. New Relic emphasizes that the views expressed in the blog are those of the author and encourages users to seek support through the Explorers Hub community.

At New Relic, programmatically tracked service level indicators (SLIs) are foundational to site reliability engineering practice, reducing manual tracking of performance and incident data. Programmatic SLIs have three key characteristics: current, automated, and useful. They help teams develop and create programmatic SLIs by identifying system capabilities, articulating plain-language definitions, defining one or more SLIs, measuring the system, defining SLOs, iterating, and refining over time. New Relic One's service level management functionality can automatically set up SLIs and SLOs, providing a one-click setup for establishing a baseline for reliability metrics. To track programmatic SLIs, teams identify existing instrumentation, deploy new instrumentation, gather metrics, create alert conditions, and create NRQL queries and dashboards to reveal when services miss their indicators. Regularly revisiting SLIs is necessary as systems evolve, and it's recommended to revisit them quarterly or whenever changes occur.

At New Relic, programmatically tracked service level indicators (SLIs) are crucial for enhancing site reliability engineering by reducing manual tracking of performance and incident data. These SLIs are characterized by their current status, automation through instrumentation, and relevance to user concerns. The process involves identifying system capabilities, defining SLIs and service level objectives (SLOs), and ensuring these metrics accurately reflect customer experience. New Relic One aids in this by offering tools to automatically or manually set up and track SLIs and SLOs, providing a baseline for service levels and enabling continuous monitoring and refinement. The platform also supports creating alerts and visualizations to ensure systems meet their reliability goals, emphasizing that SLIs should be regularly revisited as systems evolve. The integration of New Relic One's features facilitates efficient service level management, allowing teams to focus on maintaining and improving system performance and reliability.

The traditional method for load testing is time-consuming and cumbersome, requiring multiple steps such as writing test cases, building non-prod environments, and running tests. This approach has been criticized for being outdated and not suitable for modern cloud-native technologies. To address this issue, Speedscale was created to provide a new approach to scale testing using traffic replay, which enables teams to automate load testing with real traffic before deploying their applications in production. With Speedscale's traffic replay, teams can build test automation with the click of a button and get unlimited scalability without writing scripts or provisioning environments, and it integrates seamlessly with New Relic One for enhanced observability and visibility.

Traditional load testing is often complex and outdated, prompting the need for innovative solutions like Speedscale, which leverages traffic replay to test cloud-native applications effectively. Founded by Ken Ahrens and his co-founders, Speedscale simplifies the testing process by using real traffic data to create automated test environments, allowing for efficient application performance evaluation without extensive manual setup. By integrating with New Relic One, Speedscale enhances observability, providing detailed insights into application performance through dashboards that combine traffic replay results with telemetry data. This approach supports continuous integration workflows and offers significant scalability, ensuring applications perform reliably under real-world conditions. Speedscale's quickstart feature facilitates easy integration and visualization of test data, enabling development teams to address performance issues proactively.

This tutorial introduces Rails 7 and its integration with New Relic One for monitoring and observability. It guides users through creating a new Rails application, setting up New Relic One monitoring and observability, deploying the app to Render, and introducing an error feature to demonstrate the potential of New Relic One in diagnosing issues. The tutorial covers key steps such as installing the Ruby gem, downloading the configuration file, deploying to Render, and adding an error button to manually create errors for testing purposes. By following this tutorial, users can gain hands-on experience with integrating New Relic One into their Rails applications and improve their application's performance and reliability.

Rails 7 introduces a new approach to full-stack web development, featuring the ability to incorporate Node packages without Node and real-time browser interactivity via Hotwire. The tutorial guides users on integrating New Relic One with a Rails 7 application for performance monitoring, demonstrating how to deploy the application to Render, a cloud hosting provider. It includes steps to intentionally create and monitor application errors using New Relic One's dashboard, offering insights into error diagnostics and performance metrics. The tutorial assumes a basic understanding of Ruby on Rails, terminal commands, and GitHub, and it provides a complete application example on GitHub for further exploration.

New Relic has introduced a new core user type for its usage-based pricing model, designed to empower all engineers to do their best work based on data, not opinions. This new core user type unlocks telemetry data for every developer and enables them to understand code performance better, collaborate with DevOps teams, and access relevant error details directly in their IDE. The company offers three user types: basic users who can see application or system performance metrics via a shared dashboard, core users who have access to telemetry data in their IDE, and full platform users who get access to every capability within New Relic for monitoring, debugging, and improving their entire stack. Additionally, the company has made applied intelligence free for every full platform user, improved admin features to manage users dynamically at scale, and simplified pricing with core users available for all new customers.

New Relic has introduced a new pricing model and user types aimed at enhancing observability for engineers across the software lifecycle. The introduction of a new core user type, priced at $49 per month, provides developers with access to telemetry data, enabling them to optimize code performance, collaborate with DevOps, and improve feature planning directly from their IDEs. This move is part of a broader strategy that includes a usage-based pricing model and three user types: basic, core, and full platform. Full platform users now benefit from free AIOps features, which enhance anomaly detection and incident intelligence. Additionally, New Relic has expanded automated user management capabilities to all Pro edition customers, allowing for easier integration with identity providers supporting the SCIM standard. These initiatives reflect New Relic's commitment to providing comprehensive observability solutions that bridge communication gaps between development and operations teams, thereby accelerating issue resolution and improving shipping velocity.

When deploying new code, canary deployments are used to incrementally release new features to subgroups of users, reducing the risk of issues affecting all users at once. To drive these canaries safely and efficiently, this post uses New Relic to analyze application performance and detect anomalies in real-time. Argo Rollouts is a Kubernetes controller that provides advanced deployment capabilities such as blue-green, canary, and progressive delivery features. By integrating New Relic with Argo Rollouts, developers can monitor their application's performance, identify potential issues, and abort the rollout if necessary. The integration allows for proactive detection of anomalies, including errors, throughput, response time, and golden signals. With this setup, developers can test their canary releases in a controlled environment, ensuring that only healthy versions of their application are deployed to production. By leveraging New Relic's Proactive Detection and Argo Rollouts' advanced deployment capabilities, developers can make data-driven decisions about when to deploy new features, reducing the risk of introducing issues into their application.

Canary deployments, which incrementally release new features to a subset of users, can be made safer and more efficient using New Relic in conjunction with Kubernetes and Docker. This guide outlines the process of setting up Argo Rollouts, a Kubernetes controller that offers advanced deployment capabilities like canary analysis, using New Relic's AIOps Proactive Detection to monitor application health. The tutorial demonstrates configuring traffic splitting between canary and stable services via Istio, creating necessary Kubernetes resources, and setting up Argo Rollouts Analysis Templates to detect errors and anomalies using New Relic metrics. The strategy involves gradually increasing the canary's traffic while monitoring key indicators such as throughput, response time, and errors. The integration of New Relic provides real-time insights and enables aborting deployments if issues are detected, ensuring only stable versions reach full production. This example is a starting point for defining a strategy tailored to specific applications, and it highlights the importance of monitoring and analysis in successful Canary releases.

Optimize your workflow with Codestream to collaborate with colleagues easily and integrate with existing tools, prevent and fix problems with Real User Monitoring (RUM) and synthetic monitoring by combining both for end-to-end observability, harness the power of machine learning with AIOps to improve MTTD and MTTR, unlock the value of your data quickly with New Relic Instant Observability quickstarts, get notified and triage errors faster with Slack and APM integration with Errors Inbox, and consider better pricing possibilities by signing up for a free forever account.

Emphasizing the importance of starting the new year with a growth mindset, the article offers a variety of resolutions aimed at enhancing professional skills and productivity. It suggests optimizing workflow with Codestream, which integrates seamlessly with workplace tools for improved collaboration in remote settings. The use of Real User Monitoring (RUM) and synthetic monitors is recommended to enhance website performance and customer experience by providing comprehensive observability. The application of AIOps, which leverages data science and machine learning for IT operations, is highlighted for improving detection and resolution times. New Relic's Instant Observability quickstarts are introduced as a way to swiftly unlock data value through pre-built instrumentation and integration bundles. Additionally, integrating Slack with Errors Inbox is suggested for faster error triage and improved team collaboration. The article encourages exploring New Relic's competitive pricing and suggests trying their free account for a firsthand experience of their capabilities, while inviting readers to explore more resolutions through their top blogs of 2021.

New Relic's Pixie integration can be used to proactively detect and report SQL injection attempts while an application is live. A proof-of-concept script was created using Python to flag suspicious SQL queries and report them to New Relic One. The script uses a simple rule set to identify potential SQL injection queries, but the authors plan to replace this with machine learning detection in the future. The proof of concept demonstrates how Pixie can be used to detect SQL injection attempts and provides a starting point for building a more comprehensive security product using New Relic's Pixie integration.

Beyond Finance uses New Relic One for effective application monitoring to maintain resilient applications and happy customers. The team leverages features like custom events, custom dashboards, and workloads to identify and resolve incidents faster. They also utilize Errors Inbox, an error tracking solution by New Relic, to detect, triage, and resolve full-stack errors in one place, enabling better collaboration among engineers to create better customer experiences and free up time for innovation.

New Relic's Pixie integration offers a proactive approach to detecting SQL injection attempts in live applications by using a Python script that flags suspicious SQL queries and reports them to New Relic One. The focus is on monitoring rather than blocking such attempts, enabling developers to better understand their code's vulnerabilities. This proof of concept aims to integrate with the open-source Pixie project and eventually extend to other vulnerabilities like cross-site scripting (XSS) and server-side request forgery (SSRF). The long-term goal is to develop an observability-based security product using open-source components, with plans to transition from a rule set approach to machine learning detection. The script leverages New Relic's capabilities to create visualizations of security events, providing both a proactive development tool and ongoing monitoring resource.

At Data Nerd Days 2.0, Ashley Zagorski, a Software Engineer at Beyond Finance, highlighted the critical role of effective application monitoring for maintaining resilient applications and enhancing customer satisfaction through the use of New Relic One. Her team leverages New Relic One's features, such as custom events, dashboards, workloads, and Errors Inbox, to swiftly identify and resolve incidents, improve collaboration, and ultimately create better customer experiences. Errors Inbox, in particular, serves as a comprehensive tool for detecting, triaging, and resolving full-stack errors, thus enabling more efficient troubleshooting and freeing up time for innovation. The blog, authored by Scott Sugimoto, a Senior Product Marketing Manager at New Relic, emphasizes the utility of New Relic's solutions in improving infrastructure monitoring and application health, though the views expressed are personal and not representative of New Relic's official stance.