February 2021 Summaries

If you’re like most engineers, you may find yourself parsing through data from more sources than ever before. Chances are that you need a way to visualize and customize all the data to fit your organization’s needs, and report and efficiently communicate your analysis to different stakeholders. Starting from scratch isn’t the most efficient way. As a New Relic One user, you can begin accessing new dashboards and custom visualization capabilities today that give you access to a library of dashboards, third-party custom visualization capabilities, and quickstart visualizations, based on common use cases in the New Relic One Catalog and the New Relic One Dashboard API. You can now visualize your data from scratch or use pre-built templates for popular JavaScript charting libraries like D3 or Recharts, as well as maps, heat maps, and traffic lights to visualize system movements. The dashboards provide custom visualizations that can be automated through templates and a full dashboard CRUD API, with custom chart widgets built with React.js, the ability to pull in third-party library widgets, and a library of templates to get started fast—and all of it in open source. With recent customization updates, you can now quickly add widgets directly to dashboards by clicking “add to dashboards” in the Query Builder, and customize existing widgets to display data as needed. The API allows for deployment of standard sets of dashboards for every new service created, and a large inventory of pre-built custom dashboards reflect observability best practices. New Relic is continuously listening to feedback and adding more templates, resources, and visualizations to its libraries, encouraging users to join in and contribute to the development process.

New Relic is a platform for observability and full-stack observability that provides end-to-end visibility into software development trends and new product features. The Nerdlog is a weekly live stream where New Relic experts share knowledge on the latest features, including ingesting OpenTelemetry data with New Relic One, real user monitoring support for Google Core Web Vitals, data dropping via NerdGraph, agentless syslog onboarding, and other log management updates. Viewers can participate in Q&A sessions and get step-by-step instructions on how to implement these features. New Relic also offers a free account sign-up and exclusive content at the Explorers Hub for customers and non-customers alike.

Nerdlog, a series hosted on Twitch every Thursday at noon PT, offers insights into New Relic's latest observability features, with contributions from product managers and engineers. The blog recaps the second episode, covering topics like ingesting OpenTelemetry data with New Relic One, where Director of Product Management Chris Hansen explains the benefits of using OpenTelemetry for seamless integration and comprehensive visibility. Other segments include an overview of Google Core Web Vitals by Senior Product Managers Rebecca Rodriguez and Lindsy Farina, and the importance of New Relic Browser Monitoring for user experience metrics. Senior Product Manager Paul Frazier and Senior Software Engineer Ryan McMahon discuss data dropping techniques via NerdGraph, while Senior Product Manager Michael Neville-O’Neill and Senior Director Julian Giuca provide updates on log management, focusing on agentless syslog onboarding and new logs integrations. The blog encourages participation in live discussions, exploring features through NerdGraph API, and accessing early features for exporting logs, while offering readers a chance to sign up for a free New Relic account.

New Relic One users can now access advanced dashboard and visualization capabilities, enabling them to effectively parse and communicate data from various sources. These capabilities include a library of pre-built dashboards, third-party custom visualizations, and quickstart templates available through the New Relic One Catalog and Dashboard API. Users can create custom widgets using popular JavaScript charting libraries like D3 or Recharts, and deploy applications with ease using the open-source Quickstarts Library. Recent updates allow for enhanced customization options, such as formatting and axis adjustments, to tailor dashboard displays to specific needs. The platform encourages user feedback for continual improvement and expansion of its resources, emphasizing innovation and collaboration within the open-source ecosystem.

When planning a cloud migration, it's essential to consider various factors to maximize success. Establishing the migration architect role and choosing a single cloud provider or multi-cloud approach are crucial steps. Defining key performance indicators (KPIs) for monitoring application performance and establishing baselines for measuring current performance are also vital. Prioritizing migration components, performing necessary refactoring, creating a data-migration plan, and switching over production to the new environment are additional critical considerations. Reviewing application resource allocation and securing the cloud environment are also important aspects of a successful cloud migration. By following these 10 steps and considering factors such as security, cost optimization, and tooling, organizations can ensure a smooth transition to the cloud and achieve their desired outcomes.

At the heart of business innovation is software—and the teams who create it, deploy it, and operate it. Software complexity is rising, making observability a crucial tool for speed, understanding overall performance, and confidence in system changes. To address this, New Relic introduced Full-Stack Observability with a single integrated product called New Relic One, simplifying instrumenting and measuring software components. This platform now includes New Relic Explorer, an essential daily view that provides instant visibility into the entire environment, including applications, infrastructure, logs, browser, mobile—all of it. New Relic Explorer offers several new approaches to visualizing, exploring, and understanding the estate, such as real-time views, intuitive circle visualization, analysis with performance and error profiles, and estate-wide views. Full-Stack Observability customers have access to New Relic Explorer, including Pro and Enterprise customers who also get access to New Relic Lookout for advanced analysis features and capabilities.

The text provides a comprehensive guide on successfully migrating enterprise applications to the cloud, highlighting common challenges and lessons learned from past experiences to avoid repeating mistakes. It outlines a 10-step cloud migration checklist, which includes establishing a migration architect role, deciding on the level of cloud integration, choosing between single or multi-cloud strategies, and defining cloud KPIs and performance baselines. The guide further delves into prioritizing migration components, performing necessary refactoring, creating a data-migration plan, and reviewing application resource allocation, emphasizing the importance of securing cloud environments and understanding cloud costs. The text also suggests adopting modern application-building practices, such as using DevOps methods and employing tools like New Relic for monitoring cloud systems to improve application performance and customer experience.

At the core of modern business innovation, software development teams face increasing complexity, which they address by embracing observability for comprehensive performance insights and system confidence. New Relic has responded to this challenge by enhancing its New Relic One platform, integrating all existing products under Full-Stack Observability to simplify and reduce the cost of monitoring software environments. The introduction of New Relic Explorer provides software teams with an intuitive, real-time view of their entire system, offering unprecedented visibility across applications, infrastructure, and telemetry data without requiring configuration. Key features include New Relic Lookout and New Relic Navigator, which highlight crucial changes and health statuses using intuitive visualizations that allow users to rapidly identify and resolve issues. These tools aim to empower teams to build more reliable software solutions by facilitating faster issue resolution and a comprehensive understanding of their software environments.

We know you’re working hard. You don’t have time to sift through marketing announcements to understand the latest features and changes. In addition to What’s New, we wanted another way to ensure you and your teams stay informed about the newest and most essential New Relic One changes and features. Thus, the Nerdlog was born. Every Thursday at noon PT (8 p.m. UTC) on Twitch, you can learn about the latest observability features from product managers and engineers, get resources and step-by-step instructions, and engage directly with them to provide feedback and ask questions. New Relic has introduced several new features, including a ServiceNow integration that reduces noise by linking incidents, an integration with Snowflake for optimizing performance, security, and costs, and k6 load testing to manage performance in production. Additionally, you can subscribe to Nerdlog emails for weekly updates about the latest features and releases from New Relic's product team.

When planning to build a distributed application, developers must consider various patterns and technologies for intra-service communication, state management, secret stores, and securing connections. The Distributed Application Runtime (Dapr) aims to simplify this process by providing a set of building blocks that abstract specific implementations, saving developers from integrating multiple SDKs or knowing the specifics of the technology implementing these capabilities. Dapr offers benefits such as simpler code, language agnosticism, high portability, security, and observability, making it easier for developers to build and deploy distributed applications securely and instrument them for observation. With its open source project and production-ready v1.0 release, Dapr is now being used by several large and small companies to build business-critical applications.

Developers building distributed applications face various challenges beyond business logic, including ensuring effective microservices communication, integrating with external systems, managing state, and maintaining security and observability across different environments. The Distributed Application Runtime (Dapr) addresses these challenges by offering a set of building blocks that abstract specific implementations, allowing developers to focus less on integrating multiple SDKs and more on application logic. Dapr uses a sidecar pattern, making it language-agnostic and enhancing portability across different hosting technologies, as it requires no code changes when switching cloud providers. It ensures secure communication and access control, while its observability features, compatible with standards like W3C trace context and OpenTelemetry, provide automatic instrumentation and seamless integration with monitoring tools such as New Relic One. This simplifies tracing and monitoring setup, allowing developers to visualize and gain insights into their applications with minimal configuration changes.

The Nerdlog is a weekly Twitch show designed to keep users informed about New Relic One's latest features and updates, offering direct interaction with product managers and engineers for feedback and queries. Each episode covers various integrations and optimizations, such as the ServiceNow integration for managing incidents, the Snowflake integration for optimizing performance and reducing costs, and k6 load testing for enhancing system performance during production. The Nerdlog also provides resources, step-by-step instructions, and recaps for those who miss the live sessions, while encouraging users to participate in discussions and sign up for New Relic accounts to access its extensive integration capabilities.

Syslog is a standard protocol used for collecting system log messages from various machines in a central location. Open source tools like syslog-ng and rsyslog can make it easier to forward logs to centralized locations like New Relic. Rsyslog is an open source, Unix-based utility that collects, transforms, filters, and routes log messages in an IP network. It's widely used due to its flexibility and simplicity. To get started with rsyslog, users need to install the necessary packages, create a configuration file, and restart the service. The configuration can be tailored to forward specific logs to New Relic or other destinations. Disk-assisted queuing provides reliability by combining memory and disk queueing. Users can enable this feature by editing their /etc/rsyslog.conf file or creating a new file in the /etc/rsyslog.d/ directory.

Centralizing and managing log data from multiple sources can be challenging, but open-source tools like rsyslog and syslog-ng can streamline the process by forwarding logs to centralized locations such as New Relic. Syslog is a standard protocol used for sending system log messages to a specific server, supported by major operating systems like macOS, Linux, and Unix. Syslog-ng extends this protocol with advanced filtering and configuration options, while rsyslog offers a fast, open-source solution for collecting, transforming, and routing log messages, and is the default syslog utility in Ubuntu and Debian. Rsyslog is particularly favored by security-focused teams as it avoids third-party software on sensitive systems. Best practices for monitoring logs with rsyslog include centralizing log data, configuring log rotation and compression, employing reliable log forwarding mechanisms, using structured log formats like JSON for easier analysis, and setting up alerting for anomalies. Rsyslog's features, such as disk-assisted queuing, enhance the reliability and efficiency of log management, making it suitable for handling logs from networking and security devices. The article provides detailed instructions for configuring rsyslog to forward logs to New Relic, highlighting the importance of secure and efficient log data handling.

OpenTelemetry is an open source project that provides a unified standard for service instrumentation, allowing developers to collect and transfer telemetry data in a standardized way. New Relic has announced the availability of its OpenTelemetry offerings, which enable multiple ways to ingest data such as the OpenTelemetry Collector, language-specific exporters, and Java OpenTelemetry Integration Bundle. With these offerings, developers can visualize their entire stack's performance data in context, get to the root cause faster, and deliver optimum performance for their application. The project has reached version 1.0.1, a significant milestone, and New Relic plans to continue its contributions and investments to ensure that both traces and metrics for all languages reach this milestone soon.

OpenTelemetry, an open-source project backed by the Cloud Native Computing Foundation, offers a unified standard for service instrumentation by merging OpenTracing and OpenCensus, allowing for standardized collection and transfer of telemetry data to various backends, such as New Relic. New Relic has integrated OpenTelemetry into its Telemetry Data Platform, enabling users to ingest, visualize, and analyze data from distributed systems, enhancing the ability to diagnose and optimize application performance. The integration supports multiple methods of data ingestion, including the OpenTelemetry Collector and language-specific exporters for .NET, Java, and Go, providing users with tools to instrument services and send telemetry data to New Relic. With Full-Stack Observability, users can quickly identify and resolve performance issues by visualizing metrics like response time, throughput, and error rate, allowing for detailed transaction analysis and end-to-end tracing. As OpenTelemetry continues to grow, New Relic is committed to evolving its offerings to provide enhanced observability features, encouraging feedback through its Explorers Hub.

As companies navigate the shift from traditional performance monitoring to achieving end-to-end observability, they must manage organizational change and address the pain points associated with this transition. Observability requires IT leaders to communicate the "why" behind the need for change, establish clear objectives, and hold people accountable for meeting those objectives. To motivate staff, managers should focus on the customer experience, ease of use, and acknowledge trade-offs, ensuring that the benefits of observability outweigh the costs of change. By understanding these key factors, IT teams can make a smoother transition to full-stack observability and improve their ability to deliver exceptional user experiences.

Transitioning from traditional performance monitoring to end-to-end observability involves significant organizational and technological changes, driven by the increasing complexity of applications and the shift towards cloud and microservices architectures. Companies often seek observability to consolidate multiple monitoring solutions, adopt open standards like OpenTelemetry, and streamline performance management across diverse systems. Effective change management requires leadership to clearly communicate the necessity and benefits of observability, align objectives with customer-focused outcomes, and make learning and adapting to new platforms as seamless as possible. It’s essential to acknowledge the trade-offs involved, such as moving from multiple specialized tools to a unified observability platform, while ensuring that the new system offers greater long-term benefits. Managers must support their teams by emphasizing the human aspect of change, facilitating understanding, and connecting their work to improved customer experiences, ultimately making the transition smoother and more engaging for IT staff.

Cerner Corporation, a leading healthcare IT company, employs over 27,000 people and has a large engineering workforce that supports various products, including electronic medical records, data aggregation tools, and device integrations. The company focuses on observability to understand its systems and ensure continued learning from how they are used and performing in production. A principal engineer at Cerner, the author shares their experience with evolving and scaling service infrastructure for the core electronic medical record platform, Millennium, using Spinnaker, Kubernetes, and New Relic. They highlight the benefits of consolidating container build processes, instrumenting services with New Relic, eliminating toil, and driving adoption through techniques like ShipIt Days and good documentation. The company continues to collaborate with New Relic, adopting automation and building apps using their resources.

Cerner Corporation, a prominent healthcare IT company, leverages advanced observability practices to enhance its wide array of products, including electronic medical records and data analysis tools, hosted on both public and private clouds. The company has integrated New Relic's developer ecosystem into its infrastructure to streamline the instrumentation of both legacy and modern workloads, reducing operational complexity by providing consistent telemetry data across systems. This integration is facilitated through standardized container builds and the use of service profiles, enabling efficient deployment and monitoring within Kubernetes environments. Cerner's engineering teams benefit from automated alerts and dashboards generated using New Relic Terraform Provider, while ongoing initiatives like hackathons and comprehensive documentation further drive the adoption and optimization of observability solutions. These efforts aim to foster a collaborative engineering culture and improve service performance through innovative tools and methodologies.

Architects play a critical role in designing and advising on building objects, similar to conductors orchestrating musical performances. In software development, IT architects use their creative vision to design solutions and apply practical knowledge to solve problems. At New Relic, architects are expected to be technical leaders who guide engineers, make decisions that support team goals, and raise concerns to higher management levels to ensure the organization's overall acceptance of a choice. Architects must also advocate for their teams across the organization, have good communication skills, define technical standards, and engage in some technical work. To be successful, architects apply the Servant Leadership philosophy, focusing on the growth and well-being of people and communities, and measuring success based on how effectively they make their group lead and team successful.

Architects at New Relic play a crucial role akin to conductors, orchestrating the technical vision and ensuring alignment with company-wide goals. They are expected to be servant leaders, putting the organization and their teams first, and their primary outputs are leadership and influence. These professionals help engineers foresee potential challenges, make informed decisions, and avoid costly mistakes, while also elevating technical competency and serving as technical champions. They maintain technical standards to allow swift progress without compromising the system's integrity and engage in some technical work to inform decision-making. Success for architects at New Relic is measured by their ability to guide teams in problem-solving, build trust, and help navigate ambiguity, ultimately contributing to the organization's growth and well-being.

AWS Distro for OpenTelemetry is a production-ready open source distribution that enables AWS users to instrument their applications with OpenTelemetry. It includes the AWS OpenTelemetry SDKs, Java auto instrumentation agent, and OpenTelemetry collector, which provides vendor-agnostic implementation for receiving, processing, and exporting telemetry data. The collector removes the need to run multiple collectors and gives users tools to send open source observability data to any backend regardless of format. Users can run the collector in various environments such as EKS, ECS, or EC2 and configure it to export data to New Relic using OTLP exporters. By following a step-by-step guide, users can instrument their Java applications with OpenTelemetry and send data to New Relic for analysis. AWS Distro for OpenTelemetry streamlines the observability workflow, allowing users to visualize, analyze, and correlate application performance data with underlying infrastructure data, reducing mean time to problem resolution.

AWS Distro for OpenTelemetry is a secure, production-ready open-source distribution that simplifies the process for AWS users to instrument their applications using OpenTelemetry. It includes the AWS OpenTelemetry SDKs, Java auto instrumentation agent, and the OpenTelemetry collector, providing a vendor-agnostic solution for receiving, processing, and exporting telemetry data without the need for multiple collectors. The AWS OpenTelemetry Collector can be deployed in various AWS environments and allows users to send observability data to any backend. The blog post demonstrates how to configure AWS Distro for OpenTelemetry to send data to New Relic, using a sample Java application and the OpenTelemetry Collector. Users are guided through steps to download necessary components, configure New Relic as the backend, and start analyzing data with New Relic's OpenTelemetry UI. The integration allows for comprehensive visualization and analysis of application performance data correlated with infrastructure data, thereby reducing the mean time to problem resolution. The article concludes with an invitation to learn more through New Relic's developer portal and encourages signing up for New Relic's free tier for OpenTelemetry data ingestion.