November 2023 Summaries

In this blog post, OpenTelemetry is explored as a unified framework for collecting, processing, and exporting observability data from distributed systems. It simplifies the configuration and deployment of OpenTelemetry agents for Lambda functions by providing automatic instrumentation and tracing capabilities. Serverless applications face challenges in observability due to ephemeral function lifespans and lack of persistent storage or network resources. OpenTelemetry addresses these challenges with a lightweight and flexible solution, allowing developers to add minimal code changes to instrument common libraries and frameworks. The blog post focuses on the automatic instrumentation of an AWS Lambda function using Node.js, highlighting key libraries and APIs, such as `@opentelemetry/instrumentation-aws-lambda`, `@opentelemetry/sdk-trace-node`, and `OTLPTraceExporter`. By following the steps outlined in this guide, developers can gain valuable insights into the performance, behavior, and dependencies of their Lambda functions, enhancing observability with OpenTelemetry SDKs and exporting traces to New Relic.

The blog post delves into the use of OpenTelemetry for instrumenting and monitoring serverless applications on AWS Lambda, focusing on the OpenTelemetry SDK and its role in simplifying the configuration of Lambda functions. OpenTelemetry provides a unified framework for collecting observability data, including metrics, traces, and logs, which is crucial for understanding serverless applications that are ephemeral and event-driven. The post explains how OpenTelemetry supports multiple platforms and offers a vendor-neutral API and SDK for application instrumentation, along with a range of exporters for data integration with various backends like New Relic. The post provides a step-by-step guide for setting up automatic instrumentation of AWS Lambda functions using OpenTelemetry libraries and SDKs for Node.js, detailing the installation of necessary packages, SDK configuration, and environment setup. The author highlights the benefits of exporting data to New Relic, which enhances observability through service map visualizations, transactions, and alerting. The post is authored by Zameer Fouzan, who shares insights on leveraging OpenTelemetry for better serverless application performance and observability.

At Pomelo, a fintech company with years of experience in building backend infrastructure, they aim to revolutionize the industry by providing robust and reliable technology for Latin American companies to launch their financial services swiftly and simply. To achieve this, they implemented seven key steps: building an internal developer platform (IDP) called Rocket, standardizing all assets on the IDP, using one monitoring tool, mapping critical paths for each business model, decentralizing and supporting teams, introducing controlled chaos, and knowing when to prioritize speed over quality. By following these processes, Pomelo's infrastructure can scale to millions of transactions per hour without hassle, earning them a reputation as a fintech infrastructure leader in Latin America.

Apache Kafka is a powerful tool for moving large amounts of data in a fast and reliable manner, but optimizing its performance and reducing cloud costs can be challenging due to its complexity. To overcome this, it's essential to understand the costs involved and identify areas where improvements can be made. Monitoring Apache Kafka's performance using tools like New Relic is crucial to optimize network, storage, server hardware, and client configurations. By implementing a showbacks/chargebacks strategy, teams can ensure that users are using Kafka resources efficiently, reducing cloud costs in the long run. The key to success lies in understanding how different components of Kafka interact and affect each other, and leveraging monitoring data to make informed decisions about cluster configuration, application performance, and resource allocation.

Pomelo, a fintech company in Latin America, focuses on creating reliable backend infrastructure for financial services, enabling clients to launch and scale quickly. The team developed an internal development platform called Rocket, utilizing tools like Backstage, AWS, and New Relic to standardize processes and enhance observability. They prioritize a unified monitoring approach with New Relic, map critical business model paths, and promote decentralized teams to foster ownership and efficiency. Emphasizing chaos engineering and reliability, Pomelo tests systems in controlled environments to ensure robustness. While they sometimes prioritize speed over quality in non-client-facing products, the core client services maintain high standards, allowing the infrastructure to handle millions of transactions seamlessly.

Apache Kafka is a vital component in cloud architectures due to its performance and reliability in handling large data volumes, but it can also contribute significantly to cloud expenses. The article discusses various strategies to optimize Apache Kafka costs, emphasizing the importance of monitoring and understanding cost structures, particularly using tools like New Relic. Techniques include optimizing networking by fetching data from the nearest replica to reduce cross-zone traffic, employing compression to minimize storage and network usage, and increasing storage throughput to handle high I/O demands without scaling up the cluster. Additionally, server hardware costs can be reduced by optimizing CPU usage through improved batching configurations. Implementing a showbacks or chargebacks strategy, supported by comprehensive monitoring of billing, cluster, and client metrics, can further enforce efficient resource use and cost management.

Imagine a seamless online reality where every click leads to instant results and digital frustration is a thing of the past. Application performance monitoring (APM) strives to create this world by ensuring that applications run with precision, making it easier for businesses to deliver swift and smooth digital experiences that users demand. APM is like the dashboard for applications, providing a clear picture of how they're performing and helping businesses keep them running smoothly. By analyzing trends and patterns, APM can predict potential issues and allow businesses to fix them before they impact their users. This proactive approach gives businesses the superpower to ensure that their digital services are always at their best, turning potential downtimes into mere hiccups. The smooth operation of applications is crucial for customer satisfaction and business success, and APM provides a window into the health of digital services, identifying and diagnosing performance issues in real time. With APM's insights, businesses can make informed decisions about where to allocate resources to improve the user experience, leveraging predictive analytics to forecast potential performance bottlenecks and guide optimization efforts. APM tools provide key features such as real-time monitoring, troubleshooting and diagnostics, performance analytics, automation and proactive optimization, user experience tracking, application topology mapping, customizable dashboards and reporting, integration and scalability. By incorporating APM into their business strategy, businesses can safeguard their digital infrastructure, empower continuous improvement and innovation, and achieve a competitive advantage, customer delight, revenue growth, cost savings, and strategic decisions. To implement APM effectively, businesses should set clear performance goals, select the right tools, integrate APM into their development lifecycle, train their team, monitor continuously, and iterate and improve. With APM, digital services can run at their best, keeping users happy and the business sharp, by providing smooth experiences and staying ahead of the competition.

Log management is a crucial process in modern IT and DevOps practices, involving the handling of log data from its generation to archiving or deletion. Logs, which are timestamped records of events in applications, networks, or servers, are essential for troubleshooting, analysis, security, compliance, and performance optimization. Effective log management allows for centralized data collection, reducing context switching, and enabling faster problem resolution through powerful querying and visualization tools. In addition to enhancing security and compliance by providing an audit trail, logs also support incident detection and response by alerting IT teams to anomalies. They play a vital role in root cause analysis, allowing businesses to quickly identify and resolve issues, thereby improving user experience and operational efficiency. Modern log management solutions, such as those offered by New Relic, provide comprehensive features for efficient data storage, querying, and visualization, facilitating data-driven decision-making and ensuring the seamless operation of digital interactions essential for business success.

Application Performance Monitoring (APM) is a crucial tool for businesses aiming to enhance digital experiences by ensuring applications run smoothly, quickly, and without disruption. APM works like a car dashboard, providing real-time insights into application performance, helping businesses not only react to issues but anticipate and prevent them. It plays a key role in improving user satisfaction and business success by continuously monitoring applications to diagnose and resolve performance problems proactively. APM tools, such as those offered by New Relic, gather performance data to provide valuable insights into user interactions, aiding in resource allocation decisions. They also feature automation and predictive analytics to optimize applications before potential issues impact users. By integrating APM into a business's development lifecycle, training teams to utilize it effectively, and leveraging its extensive data, companies can achieve smoother digital operations, cost savings, and competitive advantages, ultimately driving customer satisfaction and revenue growth.

New Relic AI monitoring provides end-to-end visibility for any AI-powered application, allowing engineers to build and run safe, secure, and responsible AI applications with confidence. It offers a comprehensive approach to AI monitoring that incorporates technical solutions, best practices, and organizational processes. The solution brings the power of observability to engineers working on AI by providing necessary insights to debug, monitor, and improve AI applications. New Relic AI monitoring integrates seamlessly with New Relic APM 360 to provide end-to-end visibility across the entire AI stack, enabling instant insights into performance, cost, and quality issues affecting AI applications. It optimizes AI application performance, quality, and cost with deep insights, allowing for model comparison, optimization of costs, and identification of outliers and trends in responses. The solution also provides more than 50+ integrations for the AI ecosystem, including popular LLMs, machine learning libraries, vector databases, and frameworks. By adopting New Relic AI monitoring, organizations can confidently adopt AI in their applications, build trust with customers and partners, and get ahead of regulators in the ever-changing landscape of artificial intelligence.

The US government has introduced an Executive Order (EO) aimed at ensuring that Artificial Intelligence (AI) is developed and used in a responsible and beneficial manner. The EO sets forth a clear vision for the future of AI in the United States, providing a roadmap for how the federal government can work with the private sector to achieve this vision. It emphasizes three core principles: safety, security, and privacy; transparency, accountability, and responsible use; and fairness and inclusion. The EO also calls for the creation of unified standards for safe, reliable, and ethical development of AI systems. Software companies developing AI-powered applications are encouraged to adopt observability practices as a proactive step to ensure compliance, accelerate innovation, and embrace responsible AI use. The introduction of this EO marks a historic moment in AI regulation and represents a call to action for software companies to prioritize responsible AI development and deployment.

Artificial intelligence (AI) has transformed industries and interactions with technology, but it faces challenges such as inaccuracies, bias, and complexity, which are addressed in the recent U.S. Executive Order (EO) on AI. This EO, introduced on October 30, 2023, aims to ensure the responsible development and use of AI by setting guidelines centered on safety, security, privacy, transparency, accountability, fairness, and inclusion. It calls for the creation of unified standards by the National Institute of Standards and Technology (NIST) and places emphasis on AI observability practices to manage risks, enhance transparency, and ensure compliance with emerging regulations. Software companies are urged to adopt proactive measures, such as monitoring AI systems for performance and bias, to align with the new regulatory landscape and support innovation while maintaining ethical standards. While the EO primarily targets large AI vendors, smaller startups are encouraged to optimize models to avoid compliance challenges as requirements evolve. The EO represents a pivotal shift towards unified AI regulation, urging businesses to integrate observability for better compliance, innovation, and responsible AI use.

AI-powered applications present unique challenges compared to traditional apps due to their need for a specialized technology stack that includes components like large language models and vector data stores. These applications produce additional telemetry data, such as quality and cost metrics, necessitating robust monitoring to ensure safety, security, and reliability. With the Biden administration's 2023 executive order emphasizing ethical AI development, New Relic has introduced the industry's first AI monitoring solution, offering engineers unprecedented visibility across the AI stack. This monitoring solution is crucial for maintaining quality, performance, cost management, and security, addressing issues like data bias, model drift, and adversarial attacks. New Relic AI monitoring provides a comprehensive view of the AI stack, allowing engineers to debug, optimize, and ensure compliance with emerging standards. It offers more than 50 integrations with popular AI tools, enabling seamless monitoring and enhancing the observability of AI applications to build trust and stay ahead of regulatory requirements.

Application Performance Monitoring (APM) services are essential tools for understanding and optimizing the performance of applications, ensuring a superior user experience and meeting Service Level Agreements (SLAs) and Objectives (SLOs). These services, often provided by third parties, deliver critical insights into application behavior, such as latency and error causes, while facilitating automated service level management and distributed tracing. Many businesses lag in adopting APM solutions, relying instead on custom-built systems that can be costly and inefficient. Commercial APM tools, like New Relic APM 360, offer immediate benefits, such as fast time-to-value, reduced maintenance burdens, and expert support, making them a valuable investment. The decision between building or purchasing APM solutions involves assessing total costs, including hidden expenses and opportunity costs. By leveraging open standards like OpenTelemetry, organizations can avoid vendor lock-in and future-proof their operations. Essential features of APM services include easy installation, automation, integrated alerts, and advanced visualization tools to facilitate quick problem resolution and system optimization.

At D24, a worldwide payment service provider, they rely on New Relic to better understand their tech stack, discover emerging trends, and address issues before customers are affected. They use New Relic's observability features such as dashboards, custom events, canary deployment metrics, integrated alert systems, logs, and infinite tracing to create a round-the-globe, round-the-clock payments observability system. This enables them to investigate anomalies in real-time, implement solutions quickly, and notify merchants about issues to ensure the best possible experience for their end-users. By leveraging New Relic's tools, D24 has achieved an uptime of more than 99.99% without adding friction to their continuous innovation practices, resulting in a mean time to recovery (MTTR) of less than 5 minutes.

D24, a major payment service provider in Latin America, streamlines millions of daily transactions through a single API integrated with various local banks and payment methods, ensuring 24/7 availability and a service level agreement (SLA) above 99.99%. To maintain this high performance and address potential issues proactively, D24 collaborates with New Relic, leveraging its observability features like dashboards, custom events, and integrated alerts. These tools help D24 monitor real-time transaction metrics, detect anomalies, and manage traffic efficiently using methodologies such as canary deployments. The integration has enhanced D24's ability to provide frictionless payment experiences, improve mean time to recovery, and ensure robust infrastructure management, all while supporting continuous innovation and scalability.

Metadata is a crucial aspect of information technology that has been in use since 1967, initially focused on finding physical locations of data and later expanding to include logical proxies like account IDs and named environments. Telemetry data, which represents a series of numbers, can be analyzed by comparing and contrasting like or unlike groups to make informed data-driven decisions. New Relic's standard agents group telemetry under a common application name and ID, but this is not sufficient for modern observability. Metadata is essential for finding entities and telemetry, visualizing entities and telemetry, routing and responding to workflows, planning activities, and reporting business KPIs. Tags can be used to automatically apply metadata, providing a simple way to view and interact with tags in New Relic's curated views or using the entity explorer. Custom attributes can capture specific characteristics of a user's experience or application state, but come with drawbacks such as managing them centrally and incuring additional telemetry ingest cost. Lookup tables can be used to upload CSV files that relate ad hoc lookups to existing telemetry, providing a flexible solution for organizations with varying metadata needs. A simple standard for entity metadata includes five tags: Environment, Region, Team, Code Owner, and Line of Business/Business Unit, which can be modified and extended by organizations to demonstrate the value of metadata in their organization.

Metadata plays a crucial role in organizing and analyzing telemetry data in information technology, as highlighted by the history and evolution of its use from physical data location identification to logical organization through account IDs and named environments. New Relic employs metadata to enhance observability, allowing users to group and visualize telemetry data by characteristics of interest, route incident workflows efficiently, and report on business and organizational KPIs. Key strategies for managing metadata include tagging for entity organization, utilizing custom attributes for dynamic insights, and employing lookup tables for ad hoc analysis, which together enable more actionable data. Tags, which can be automatically or manually applied, facilitate incident routing and team accountability, while custom attributes offer granular differentiation for telemetry records. Lookup tables, uploaded as CSV files, provide a flexible way to associate metadata with telemetry, particularly in project tracking and cloud migration scenarios. Despite the potential challenges of managing metadata, such as query performance issues and the risk of obsolete attributes, these approaches offer significant benefits in terms of data-driven decision making and operational efficiency.

New Relic's Session Replay offers a unique perspective on frontend glitches by capturing user actions and providing a visual context that traditional debugging tools lack. This allows developers to identify and troubleshoot issues more effectively, while also optimizing user experience and improving customer retention. By setting up Session Replay, teams can gain a deeper understanding of user behavior, collaborate across functions, and resolve issues faster. The tool provides a single platform for cross-functional teams, fostering collaboration and a shared understanding of user needs. With its advanced features and customizable settings, Session Replay empowers developers to proactively identify pain points and optimize the user journey, creating a better web experience that keeps users happy and coming back for more.

New Relic's Session Replay, now generally available, offers a comprehensive tool for decoding frontend glitches by providing an immersive view of user interactions. As part of New Relic's Digital Experience Monitoring (DEM) solution, Session Replay captures 100% of user actions leading to significant incidents like errors or crashes, offering a visual context that traditional DEM tools lack. This capability allows cross-functional teams, including application developers, product managers, UI/UX designers, and customer support, to collaboratively troubleshoot and optimize user experiences by observing user journeys in detail. Enhanced privacy controls and options for extended data retention ensure compliance while enabling effective session analysis. By using New Relic's browser agent, teams can set up Session Replay to capture essential frontend data, allowing them to monitor and debug issues efficiently, optimize user flows, and provide contextual support. This tool not only aids in identifying and resolving frontend errors but also enhances the overall user experience by offering insights into user behavior, enabling data-driven decisions to reduce friction points and improve customer satisfaction.