February 2022 Summaries

Log monitoring for cloud-native architectures presents unique challenges due to the complexity of modern web applications, which source data from microservices, use Kubernetes and other container technologies, and integrate open-source components. The best practices for log monitoring include implementing a centralized log management solution, adopting open standards such as OpenTelemetry, embracing the latest tracing and logging technologies, and logging only what is necessary to provide enough context for analysis. Additionally, cloud-native environments require specialized solutions to address challenges such as scale, ephemeral storage, log variety, vendor lock-in, and security compliance issues. Implementing a full-stack observability tool like New Relic can help teams gain real-time visibility into application behavior, respond quickly to issues, and continuously improve performance and reliability across their stack.

Log monitoring for cloud-native architectures necessitates a revised approach due to the complexity of modern applications that incorporate microservices, Kubernetes, and various open-source components. These systems generate a vast volume of logs that require efficient aggregation, analysis, and storage strategies to ensure effective application health monitoring. Key challenges include managing large-scale log data from numerous microservices, addressing ephemeral storage issues in containerized environments, and handling varied log formats from diverse cloud services and APIs. To overcome these challenges, best practices include implementing centralized log management solutions, adopting open standards like OpenTelemetry to avoid vendor lock-in, and utilizing advanced tracing and logging technologies such as eBPF. It is also crucial to log only the necessary metadata to avoid data overload and ensure logs are actionable. Specialized observability platforms, like New Relic, offer comprehensive solutions for real-time monitoring and performance optimization, providing teams with the tools needed to address the specific challenges of cloud-native environments, such as those found in Google Cloud Platform and Amazon Web Services.

New Relic has achieved racial and gender pay equity by analyzing its compensation rates with a third-party company, Syndio, and implementing changes to simplify job architectures and provide guidance to managers on raises to address discrepancies. The company aims to maintain pay equity through ongoing analysis and continues to focus on driving systemic change and increasing representation of women and minorities in technical and leadership roles.

New Relic has achieved racial and gender pay equity by ensuring that employees with similar experience and job roles receive equal compensation, regardless of gender or ethnicity. This accomplishment is part of their broader Diversity, Equity, and Inclusion (DE&I) strategy, which was supported by working with Syndio, a pay equity software and consulting company, to analyze pay disparities. The company has taken steps such as simplifying job architectures and providing manager guidance to address discrepancies. Furthermore, New Relic is committed to maintaining pay equity through quarterly analyses and is focused on increasing the representation of women and individuals of color in technical and leadership roles. The company's new DE&I website provides transparency by regularly updating workforce demographic data to track progress.

New Relic One is a monitoring and observability tool that helps developers identify performance issues in their production applications. The tool can be integrated with CodeStream, an IDE extension, to provide direct access to the source of errors in the code. This integration allows developers to quickly diagnose and fix issues without leaving their IDE. With New Relic One, developers can track telemetry data, detect anomalies, and set up error monitoring for their Elixir applications, including those built with Phoenix. The tool provides a free account option that includes 100 GB/month of free data ingest, one free full-access user, and unlimited free basic users.

Elixir, a programming language introduced in 2011, has gained significant popularity, especially with its Phoenix web framework being used in applications like Discord and the Financial Times. Despite its performance advantages, developers need to monitor their Elixir applications for potential issues, which can be efficiently managed using tools like New Relic One and CodeStream. This tutorial explains how to enhance developer productivity by using CodeStream within New Relic One to identify and address performance anomalies in Phoenix applications. It guides users through creating a web application, deliberately introducing a bug, and using CodeStream to trace the issue directly in the integrated development environment (IDE). The tutorial requires familiarity with Elixir, Phoenix, terminal commands, GitHub, and a New Relic One account, offering step-by-step instructions to set up a Phoenix application with TailwindCSS, deploy it to a cloud service like Heroku, and integrate New Relic One for monitoring. Through this process, developers learn to utilize New Relic's observability features to diagnose and resolve errors, highlighting the seamless integration of CodeStream for efficient debugging.

OpenTelemetry is a community-contributed instrumentation standard that enables standardized telemetry data collection from applications and services without vendor lock-in. New Relic now provides an enhanced OpenTelemetry offering with support for OpenTelemetry Protocol (OTLP) over HTTP/1.1, infinite tracing, and curated experiences for ingested data, enabling faster troubleshooting and a seamless transition to OpenTelemetry. To get started, try out OpenTelemetry with your own applications or use the New Relic OpenTelemetry examples. Once instrumented, ingest data into New Relic One using the native OTLP endpoint, which requires only a few steps. View OpenTelemetry data through New Relic's dynamic UI, group, facet, and filter metrics, and utilize curated views for troubleshooting. New Relic Explorer provides real-time monitoring of environments, visualizing emerging issues in real time without relying on static thresholds or dashboards. AutoMaps help identify performance problems, while tracing errors enables root cause analysis with error details and stack traces. The OpenTelemetry offering is ever-evolving, empowering engineers to harness the complete power of New Relic One regardless of instrumentation source.

OpenTelemetry, a community-driven framework for standardizing telemetry data collection, provides vendor-neutral solutions that enhance data monitoring and troubleshooting capabilities. New Relic's integration with OpenTelemetry supports the OpenTelemetry Protocol (OTLP) over HTTP/1.1 and facilitates seamless transitions to OpenTelemetry, offering infinite tracing and curated experiences for ingested data. Users can employ New Relic's explorer tools like Lookout and Navigator to visualize and manage OpenTelemetry data across systems, enabling real-time insights and efficient issue resolution. New Relic also provides automaps for tracing dependencies and performance issues within distributed architectures, alongside tools for error and log correlation. As OpenTelemetry evolves, New Relic continues to enhance its support, aiming to empower engineers to optimize application performance by utilizing comprehensive telemetry data.

New Relic is making its new infrastructure monitoring experience generally available to help DevOps, SRE, and IT Operations teams identify and resolve issues in their cloud infrastructure more efficiently. The new experience delivers usability and efficiency with end-to-end platform context, enabling engineers to troubleshoot when issues arise. It addresses common problems such as detecting, triaging, and troubleshooting health issues in hybrid cloud, workload containerization, and microservices-based architectures. With this new interface, teams can pinpoint infrastructure bottlenecks, quantify impact radius, investigate root cause of anomalies, and achieve full-stack visibility with real-time troubleshooting workflows and unique topology visualizations. The experience is now available to all New Relic users without additional cost, and can be accessed in the main menu under Infrastructure > Hosts New.

New Relic has launched a new infrastructure monitoring experience, aiming to assist DevOps, SRE, and IT Operations teams in efficiently identifying and resolving issues in diverse cloud environments. This advanced monitoring tool provides unified visibility across infrastructure components and leverages features like New Relic Lookout for proactive issue detection, automap for visualizing impact radius, and timewarp for analyzing historical telemetry. By offering capabilities to pinpoint infrastructure bottlenecks and investigate the root causes of anomalies, New Relic enhances real-time troubleshooting and system optimization. The platform integrates with cloud services such as AWS, Azure, and Google Cloud, and continues to support Kubernetes cluster monitoring and Pixie for no-code observability, ensuring comprehensive stack visibility.

Amazon CloudWatch Metric Streams can be used to detect and quickly respond to operational events by integrating with New Relic, providing a unified solution for deep application and infrastructure monitoring. The AWS Well-Architured Framework helps design and operate reliable, secure, efficient, and cost-effective workloads on AWS, while understanding telemetry data from AWS components is crucial for system state visualization during operational events. By creating an Amazon Kinesis data firehose delivery stream, enabling integration with New Relic, setting up policies to alert, testing the setup, and exploring the Metric Streams data, users can automate the process of integrating CloudWatch metrics with New Relic, eliminating the need to build custom integrations for each AWS service.

The New Relic Netlify Plugin simplifies monitoring Jamstack application performance with real-user monitoring (RUM) and provides critical insights into site performance metrics in production. The plugin automates the installation of the browser monitoring agent, notifies New Relic One when a build event occurs in Netlify, and measures site performance using Google Core Web Vitals. It also allows developers to quickly discover and fix errors before they impact users, explore data using custom-built dashboards, and receive notifications when there are issues with their Netlify site. The plugin can be easily installed via the Netlify UI, by creating environment variables, or by adding a configuration block to the netlify.toml file. Additionally, developers can customize alerts for performance in New Relic One and explore additional features and configuration options in the plugin documentation.

The Cloud Native Computing Foundation (CNCF) Annual Survey Report highlights the significant growth and maturity of Kubernetes in a short period. The report shows that Kubernetes has become a viable solution for real-world problems, with widespread adoption across various industries and use cases. Developers and organizations are maturing in their approach to containerization, recognizing the value of comprehensively refactoring applications to take advantage of container technology. As a result, there is a growing demand for consistent orchestration, scalability, and observability, which Kubernetes excels at providing. However, the report also reveals that many organizations prefer using managed service providers to handle Kubernetes infrastructure, which could lead to users not understanding the underlying technology. Despite this challenge, the author believes that Kubernetes will continue to shine and reign supreme in its space due to its commoditization and maturation of its user base.

Netlify simplifies build and deployment workflows for developers, and New Relic has introduced the New Relic Netlify Plugin and quickstart to enhance monitoring of Jamstack applications by integrating with Netlify. This plugin allows developers to automatically install browser monitoring agents and receive notifications in New Relic One with each build event, providing insights into real-user performance metrics and Google Core Web Vitals. Installation can be done via Netlify UI or by adding code to the netlify.toml file, with configuration through environment variables. Alerts and customizable dashboards in New Relic One help monitor and address performance issues. Users are encouraged to explore additional features and configurations in the plugin documentation, and New Relic offers free accounts with data ingest and user access options. The blog post emphasizes that the views are those of the author and encourages interaction through New Relic’s Explorers Hub for support.

Kubernetes has rapidly evolved from a niche technology to a ubiquitous solution for managing containerized applications, as highlighted in the Cloud Native Computing Foundation's Annual Survey Report. The report, supported by data from New Relic and Datadog, underscores the widespread adoption of Kubernetes for production workloads, especially in public cloud environments and by non-SaaS companies, reflecting a maturation in containerization practices. This growth is paralleled by an increasing preference for managed Kubernetes services, indicating organizations' desire to leverage Kubernetes without delving into its complex intricacies. However, this shift also presents challenges, as it could lead to a lack of understanding of the underlying systems, similar to other commoditized services like electricity and plumbing. The New Relic O11y Trends Report for 2022 further emphasizes Kubernetes' popularity, with a significant portion of IT decision-makers actively exploring or deploying Kubernetes, although this widespread adoption necessitates enhanced observability and security measures. As organizations increasingly behave like cloud-native entities, adopting CI/CD practices and robust observability solutions, they not only benefit from improved efficiency but also contribute to advancing the cloud computing landscape.

New Relic is launching a new version of its external services dashboard to help developers rapidly identify issues with APIs and microservices that power their applications. The new dashboard features a map view for a high-level overview of all services, allowing users to quickly see which services are healthy and which are causing issues. A list view provides more detailed information on each service, including response time, call count, and error rate. Additionally, the new version is powered by distributed tracing, which can gather more useful data than cross-application tracing, helping developers understand how requests traverse their microservice-based applications.

External services are integral to modern applications, enabling seamless interactions between smart devices and applications, yet they pose challenges when issues arise, as identifying the problematic API or service can be complex. New Relic addresses these challenges with a new version of its external services dashboard, offering a map view for a visual overview of service health, dependencies, throughput, and response times, as well as a list view for sorting services by performance metrics. Enhanced by distributed tracing, this tool provides comprehensive data across various services, aiding in pinpointing issues within microservice-based applications. New Relic invites users to explore these features with a free account, offering significant data capacity and user access, while also emphasizing that the author's views are personal and not representative of New Relic's official stance.

At New Relic, defining and setting service level indicators (SLIs) and service level objectives (SLOs) is crucial for site reliability engineering (SRE) practice. The company uses a simplified version of its architecture to illustrate how to apply SLIs and SLOs in a real-world complex modern software platform. By focusing on system boundaries, New Relic simplifies the measurement process and captures the value of critical system measurements. The team establishes a baseline for service boundaries with one click, sets SLI + SLO using a simple recipe, and measures customer experience to understand SLO/SLIs for UIs. Hard dependencies require higher SLOs, such as the network tier, which has a significant impact on the overall platform's reliability. By defining SLIs and SLOs for specific capabilities at system boundaries, combining them into single SLOs, documenting and sharing contracts, and assuming evolution over time, New Relic aims to help teams build resilient and reliable software architectures.

At New Relic, setting Service Level Indicators (SLIs) and Service Level Objectives (SLOs) is crucial for effective site reliability engineering (SRE), providing a framework to assess system availability and performance across complex software architectures. The post explains how New Relic uses a simplified version of their architecture to demonstrate the practical application of SLIs and SLOs, emphasizing the importance of defining these metrics at system boundaries rather than individual components to efficiently manage reliability. The article guides readers through the process of setting SLIs and SLOs using the New Relic platform, highlighting its automated tools for establishing baselines and customizing SLIs/SLOs. It stresses the importance of clear definitions for availability, utilizing plain-language metrics, and prioritizing customer experience, particularly for UI interactions. Additionally, the network tier is emphasized as a critical dependency requiring higher SLOs to ensure overall platform reliability. The piece concludes with a reminder of the iterative nature of reliability practices and encourages leveraging New Relic's tools to streamline the setup of service levels.

This integration between Lacework and New Relic enables organizations to identify and track potential log4shell vulnerabilities in their systems, even if they are not directly exploiting the vulnerability. Lacework's approach is based on finding anomalies in system behavior rather than hardcoding specific attack details, making it more effective against emerging threats. The integration allows users to gauge their risk of being vulnerable to log4shell exploits and detect potential exploitation attempts, providing a layered defense against security threats. By setting up the integration, existing Lacework and New Relic customers can quickly identify security anomalies and take proactive measures to protect their systems.

The blog post discusses the challenges and solutions associated with the log4shell zero-day vulnerability, highlighting the integration of Lacework and New Relic to enhance security measures. While it avoids a detailed explanation of the vulnerability itself, it emphasizes the importance of being proactive against potential threats and the benefits of using cloud observability tools. The integration allows users to identify if their systems are vulnerable and whether they have been compromised, focusing on detecting anomalies rather than specific attacks. The post also mentions an upcoming webinar by Lacework and New Relic to discuss the industry's current state and strategies for dealing with security threats. Lacework's approach does not require updates for specific zero-days, as it continuously monitors systems for unusual activities, offering a robust security framework that remains relevant for future exploits. The setup process for the Lacework and New Relic integration is described as quick and user-friendly for existing users, aiming to provide peace of mind against evolving cyber threats.

New Relic One offers a wide range of ways to send telemetry data, including APM agent, browser agent, mobile agent, infrastructure agent, Kubernetes agent, on-host integrations, cloud integrations, OpenTelemetry, Metric API, Prometheus, Event API, Trace API, Edge with Infinite Tracing, log forwarding, Log API, serverless logs and metrics, Build your own integration with New Relic Flex, Telemetry SDKs, Observability as code, and Proactive anomaly detection. These methods provide flexibility in processing telemetry data to gain a comprehensive view of system performance and behavior.

Observability in software systems involves understanding system states through external outputs, which can be challenging with complex and distributed systems. New Relic One offers a flexible observability solution that processes telemetry data to provide a comprehensive view of system activities. It supports various methods for sending telemetry data, such as APM agents, browser and mobile agents, infrastructure and Kubernetes monitoring, cloud integrations, OpenTelemetry, and APIs for metrics, events, and logs. Additionally, New Relic provides tools like New Relic Flex, Telemetry SDKs, and observability-as-code with Terraform to enhance data collection and monitoring. The platform also features proactive anomaly detection using machine learning and offers a free account with data ingest and user access, helping organizations achieve comprehensive observability for their systems.

There's a difference between your job title and the skills you bring to the table. Your value as an IT practitioner lies in your problem-solving abilities, knowledge of tools, and ability to manage environments, not just the specific tool or application you use. Defining yourself by tool can diminish your actual accomplishments and make it harder for others to see your unique strengths. Instead, focus on developing a set of competencies that help teams solve problems, and be open to learning new skills and techniques. This approach will help you protect yourself from boredom and stagnation, and also ensure that you're not gatekeeping qualified candidates from the industry. By doing so, you'll become a more interesting and dynamic person in the tech world, and have a wider range of opportunities for growth and development.

Many IT professionals define themselves by the tools and vendors they frequently use, such as "Splunk engineer" or "Microsoft admin," which can diminish their broader accomplishments and potential. While it's natural to take pride in specific skills, the value of an IT professional lies in their problem-solving abilities, adaptability, and continuous learning, rather than their expertise with a specific tool that may eventually become obsolete. The article advocates for a broader self-identification based on competencies and problem-solving abilities, which are crucial for long-term career growth and can help new entrants feel less intimidated by industry-specific labels. It emphasizes the importance of lifelong learning and adaptability in the tech industry, suggesting that professionals should focus on being dynamic and open to new ideas and tools, such as integrating monitoring and observability telemetry into their work. The author, Leon Adato, who has extensive experience in IT, encourages individuals to seek workplaces that recognize their full potential beyond a vendor-specific skill set and to engage in community efforts that lower barriers to entering tech careers.