September 2022 Summaries

Apollo Server is a popular framework for building APIs based on GraphQL, a query language for APIs that includes a server-side runtime for executing queries. The flexibility of GraphQL comes with a performance cost when running complex queries against the Apollo server. This blog post will guide you through using OpenTelemetry and New Relic to measure the performance of both federated and non-federated Apollo at every phase of the query cycle, providing curated dashboards, tracing details, and gateway-specific spans. The setup involves installing required libraries, creating an `open-telemetry.js` file, integrating New Relic, and configuring OpenTelemetry instrumentation for subgraphs or monolithic applications. For federated Apollo, additional configuration files are needed to integrate the gateway into the architecture. With this integration, you can analyze your Apollo Server data in New Relic using NRQL queries, debug issues with the OpenTelemetry integration, and take advantage of features like service maps and query resolution details.

Apollo Server, a popular framework for building GraphQL-based APIs, can experience performance challenges with complex queries due to its inherent flexibility. This blog post guides users on leveraging OpenTelemetry and New Relic to monitor Apollo Server's performance by tracing every phase of the GraphQL query cycle—parsing, validating, and executing queries. It explains how to set up OpenTelemetry with Apollo Server for both federated and non-federated configurations, and how New Relic's dashboards provide detailed insights into query performance, including trace counts, durations, errors, and more. The post provides step-by-step instructions for installing necessary libraries, configuring OpenTelemetry, and sending tracing data to New Relic, with specific considerations for federated Apollo setups that combine multiple GraphQL APIs into a supergraph. Additionally, it offers debugging tips for the OpenTelemetry integration and encourages users to explore New Relic's query capabilities for deeper analysis. The article is authored by Mat Wilk, a software engineer consultant for New Relic, and emphasizes that the views expressed are personal and not representative of New Relic's official solutions.

New Relic's domain capture feature allows companies to register their domains with the platform, enabling new users signing up with a matching email domain to automatically join their existing organization. This alleviates stress from manual oversight and provides seamless access to company telemetry data for new team members. To set up domain capture, account administrators must have access to organization settings, add the domain to the platform, and configure auto provisioning preferences. The feature is available only for Pro or Enterprise edition customers. By registering their domains, companies can improve user governance, reduce troubleshooting challenges, and enhance collaboration among existing New Relic users.

New Relic has introduced a domain capture feature designed to streamline the onboarding process for new users and mitigate issues related to creating separate accounts within a company. This feature allows companies to register their email domains with New Relic, ensuring that new users who sign up with matching email domains are automatically directed to join the existing company organization, thereby reducing administrative burdens and enhancing collaboration. To set up domain capture, users must have access to organization settings within the Pro or Enterprise edition of New Relic and follow a verification process involving DNS records. This feature aims to improve user governance, centralized identity management, and system-wide data access, addressing challenges faced by IT Ops and DevOps teams.

Managing high cardinality data is crucial for optimizing resource usage and performance in Prometheus monitoring systems, especially when dealing with cloud-native environments that can experience sudden spikes in cardinality. High cardinality leads to increased memory usage, storage requirements, query latency, resource intensiveness, and scalability challenges. To manage high cardinality data from Prometheus, it is essential to set up scrape jobs with meaningful labels, reload configurations without restarting the server, check existing cardinality, view most expensive jobs, omit scrape jobs from remote write, reduce cardinality of a scrape job by dropping high cardinality labels or datapoints entirely, and store data for shorter periods. By implementing these strategies, users can strike a balance between gathering detailed data and managing costs in their Prometheus monitoring systems.

Managing high cardinality data in application monitoring is crucial for balancing detailed insights and storage costs, particularly when using tools like Prometheus and New Relic. High cardinality provides valuable granularity in data, such as unique identifiers in distributed applications, but can lead to increased resource consumption and performance issues, including higher memory usage, storage demands, and slower query performance. The blog post discusses strategies to manage high cardinality, such as configuring Prometheus with meaningful labels, using the Prometheus remote_write configuration, and employing New Relic's facilities to drop unnecessary data points or labels. It also suggests minimizing long-term storage costs by manipulating data retention settings in New Relic, allowing for real-time analysis without long-term storage expenses. The post emphasizes the importance of understanding and managing cardinality to ensure efficient monitoring, resource usage, and scalability in cloud-native environments.

According to the 2022 Observability Forecast report, many bosses are already advocates for observability, with 66% perceiving directors, VPs, and executives as strong supporters. However, some leaders may need convincing, especially if they resist or aren't aware of observability. To make a case for observability, one can emphasize the importance of monitoring the entire tech stack, connect it to the bottom line, and discuss the impact of tool and data fragmentation. The report highlights that full-stack observability is still a rare achievement, with only 27% of organizations having achieved it. Despite challenges, the benefits of observability are clear, including increased efficiency, business growth, reduced staff burnout, and improved customer experience. With many companies expanding their observability capabilities, it's essential to share insights and best practices to help leadership prioritize this critical aspect of application performance and customer experience.

Observability is increasingly recognized as essential for understanding application performance and enhancing customer experience, with the 2022 Observability Forecast report highlighting that a significant portion of IT decision-makers already advocate for its adoption. The report reveals that while many organizations support observability, only a small percentage have achieved full-stack observability, indicating a gap in comprehensive monitoring across the tech stack. Despite the fragmented use of multiple monitoring tools, there is a strong preference for consolidating data into a single repository to improve efficiency and goal achievement. The report also underscores the financial feasibility of adopting observability practices, noting that organizations achieving full-stack observability allocate a modest portion of their IT budgets toward it. The benefits of adopting observability practices include improved uptime, reliability, customer experience, and business growth, with many companies planning to expand their observability capabilities in the coming years.

CDNs are powerful tools for scaling reach and optimizing user experiences, but can become difficult to diagnose issues with during high-capacity events. Monitoring CDN performance is critical to protecting revenue and customer trust. New Relic has partnered with leading CDN providers to create quickstarts that bundle integration instructions, pre-built dashboards, and alerts to help users start monitoring their CDN's performance in minutes. These integrations enable users to track important performance metrics such as load balancing, latency, availability, and other information in the context of all telemetry data. The integration process varies by CDN provider, but typically involves creating a job or stream in the CDN control center and configuring the destination to send logs to New Relic. With these integrations, users can quickly explore their data to resolve CDN-related issues faster and gain deeper visibility into both application and platform performance data.

Monitoring Python applications is crucial to ensure optimal performance and identify bottlenecks quickly. There are four layers to consider: presentation layer (user interface), business layer (application logic), persistence/database layer (database interaction), and metrics collection (data analysis). Monitoring tools can provide insights into response time, throughput, error rate, CPU usage, memory usage, Apdex score, and other key metrics. Distributed tracing allows tracking requests through the system to identify issues in complex applications. Open-source tools like OpenTelemetry, Prometheus, Jaeger, Zipkin, logging, and structlog are available for monitoring Python applications. However, using a consolidated platform like New Relic can simplify monitoring by including automatic instrumentation, built-in visualizations, real user monitoring, synthetic monitoring, distributed tracing, and more.

New Relic has enhanced its Kubernetes monitoring capabilities by introducing a unified user interface that integrates application and cluster performance data, allowing users to efficiently identify and resolve issues within Kubernetes environments. This updated experience, now available in New Relic's application performance monitoring (APM), provides real-time activity streams, one-click logs, and the ability to correlate performance data between applications and clusters, which is crucial for developers who need to understand how cluster performance impacts their applications. By consolidating telemetry data and linking metadata, users can better investigate anomalies and performance changes, such as memory utilization and network errors, through a single interface without navigating between multiple tools. The new features aim to streamline the troubleshooting and optimization processes, making it easier for teams to collaborate and focus on critical issues, while also offering free access to new users as part of New Relic's platform offerings.

Laravel is a free, open-source PHP web framework that has been under active development since 2011 and is one of the most popular PHP frameworks with a robust community and wide range of features. Application performance is critical for web applications, making Laravel a strong choice for web development. Optimizing Laravel performance is crucial to growing user base, keeping users happy, and growing business. Factors that can negatively affect application performance include poor code quality, inefficient server configurations, and insufficient resources. Measuring performance involves utilizing monitoring tools such as New Relic, which provides detailed insights into various aspects of the application. Optimizing performance involves updating PHP and Laravel versions, debugging and optimizing with Laravel Debugbar, configuring error handling, considering Laravel Octane, utilizing route caching, using database caching, leveraging queues, generating class maps with Composer, leveraging dependency injection, monitoring and optimizing database transactions, using rate limiting, setting up baseline performance monitoring and proactive alerting. Optimizing Laravel performance improves user experience, faster content delivery, increased business revenue, reduced workload, and using New Relic to improve performance.

The 2022 Observability Forecast report highlights the growing importance of observability for IT decision-makers and organizations, with over three-quarters of respondents citing C-suite executives as advocates of observability. The study found that organizations prioritize full-stack observability to reduce outages and improve mean time to detection (MTTD) and resolution (MTTR), and nearly half expect to deploy a single, consolidated observability platform by 2025. Despite concerns over cost, the benefits of observability are deemed worth the investment, with most respondents expecting to increase their observability budgets next year.

Next.js is a modern JavaScript framework that optimizes page and application speed, development speed, and experience. To monitor its performance, you need to set up application performance monitoring for both the server side and browser monitoring. This involves installing the New Relic Node.js APM agent and middleware for Next.js, configuring your `newrelic.js` file, and integrating the New Relic Browser agent into your `_document.tsx` file. With this setup, you can view your application's performance data in New Relic, including server-side metrics such as Apdex score, throughput, and transactions, as well as browser-related metrics like largest contentful paint, input delay, AJAX requests, or page views. Additionally, you can send detailed error information to New Relic using the `_error.tsx` file, which captures both backend and frontend errors and sends them to New Relic's noticeError method. The setup also enables automatic logs in context options, allowing for correlated logging between transactions and logs. By following these steps, you can gain full-stack observability in your Next.js application and optimize its performance.

Next.js, a modern JavaScript framework, enhances application speed and performance but requires monitoring to manage performance issues and errors effectively. This blog post explores setting up full-stack observability for Next.js applications using New Relic, beginning with backend monitoring through the New Relic Node.js agent and middleware, which is integrated by default in the agent's latest version. This integration focuses on server-side rendering and transaction monitoring, providing comprehensive observability of server-side activities. For frontend monitoring, the New Relic Browser agent is injected to gather browser-related metrics, enhancing insights into user behavior and performance. Additionally, error reporting is streamlined by configuring the _error.tsx file to send both backend and frontend error details to New Relic. Log contextualization is also supported, allowing automatic correlation of application logs with telemetry data using frameworks like Winston. The article highlights the importance of monitoring both server and client sides to ensure robust performance and user experience in Next.js applications.

With the release of Amazon Virtual Private Cloud (VPC) Flow Logs via Amazon Kinesis Data Firehose to New Relic, users can now easily monitor and troubleshoot network connectivity issues across their AWS estates. VPC flow logs provide key insights for performance analytics, enabling engineers to quickly identify performance bottlenecks and security issues. The integration simplifies the delivery of VPC flow logs, reducing costs and pipeline complexity, and allows users to explore data from multiple accounts and regions in a single view. With this integration, users can gain cloud-scale observability, monitor network traffic, visualize performance metrics, and diagnose issues using curated insights and visualizations.

Amazon Web Services (AWS) has announced support for Amazon Virtual Private Cloud (VPC) Flow Logs via Amazon Kinesis Data Firehose, facilitating the integration with New Relic to enhance network performance analytics and troubleshooting. This integration allows users to monitor and visualize network traffic from within their VPCs, providing insights into metrics such as bytes and packets per second, and helps diagnose security and connectivity issues. New Relic's platform enables users to explore VPC performance across multiple accounts and regions, using tools like New Relic Explorer, Navigator, and Lookout to visualize network data and identify potential problems quickly. The integration streamlines the delivery of VPC flow logs, reducing costs and complexity, and supports both AWS CLI and CloudFormation for configuration. The collaboration between AWS and New Relic aims to accelerate cloud innovation and migration while providing a comprehensive view of network telemetry to improve customer services, as highlighted by industry professionals like Dani Konstantinovski from Gett. With the New Relic system, users can efficiently pinpoint network issues, leveraging curated metrics and alerts to maintain optimal network performance.

The evolution of web development has led to increased complexity and expectations from applications to be available on multiple devices, resulting in a need for more sophisticated tooling and infrastructure, including containerization, distributed architecture, and Kubernetes. This growth in complexity also made it challenging to obtain telemetry data, leading to the creation of OpenTelemetry, an umbrella project that unified OpenTracing and OpenCensus into a single standard for metrics and traces across all application surfaces. OpenTelemetry offers a unified API for collecting and shaping metric data, reducing the need to learn unique telemetry API specifications and reformatting data for new observability providers, thereby decreasing complexity and cognitive overhead.

Web development has evolved significantly from its simpler days of embedding PHP in HTML, largely due to increasing demands for applications to perform complex functions across various devices, which has led to the rise of containerization and distributed architecture managed by tools like Kubernetes. This complexity has also made obtaining telemetry data more challenging, prompting the development of OpenTelemetry (OTel) by the Cloud Native Computing Foundation, which unified the earlier OpenTracing and OpenCensus projects to provide a vendor-neutral standard for collecting and exporting telemetry data. OpenTelemetry simplifies the integration of metrics and traces across applications, allowing engineers to send data to any backend without needing to learn new API specifications or change dependencies for different vendors. The use of OpenTelemetry reduces cognitive overhead and tooling complexity, as demonstrated by its integration with platforms like New Relic, where engineers can easily configure their applications to export telemetry data using consistent protocols.

As the software industry shifts its focus towards environmental, social, and sustainability practices, new reporting regulations have emerged, including the EU's Due Diligence proposal and Taxonomy for Sustainable Activities. To track data across their entire supply chain, companies must build or buy an ESG dashboard that utilizes data visualization and regulatory reporting data, offering options to customize data taxonomies and categories to meet specific needs. Common features in ESG dashboards include automated data collection, portfolio-level insights, benchmarking against peers, compliance with EU regulations, and sustainability taxonomy. Companies can also opt for pre-made ESG dashboards with taxonomy definitions and category matching for supplier data in place. To ensure regulatory compliance, companies must monitor metrics such as CPU and memory usage, API consumption, response time, data volume, corrupted data, data invariance, and training and serving features. By measuring these metrics, companies can optimize their ESG dashboards and improve sustainability reporting transparency.

As the software industry increasingly focuses on sustainability, new reporting regulations, such as the EU's Due Diligence proposal and Taxonomy for Sustainable Activities, require companies to track and communicate their environmental and social impacts. Companies can either build or buy ESG dashboards to monitor their supply chain data and ensure compliance with these regulations. Building a custom dashboard allows for tailored data categorization but requires more initial effort, while off-the-shelf solutions offer pre-defined features and ease of use. Key features of ESG dashboards include automated data collection, benchmarking, compliance with sustainability regulations, and maintaining data privacy and protection. Monitoring metrics like CPU usage, API performance, data volume, and machine learning operations help ensure accurate and efficient data tracking. As these regulations become more prevalent, ESG dashboards will play a crucial role in helping companies meet their sustainability goals.

Every time customers interact with a company’s applications and services they expect great experiences. Digital services should be available, fast, and easy to use. But not every company has the right observability practices in place to measure and improve digital customer experience in a meaningful way. To better customer experience, companies can start by measuring the right key performance indicators such as availability, largest contentful paint, first input delay, cumulative layout shift, time to first byte, AJAX response times, HTTP error rate, and JavaScript error rate. They should also segment their data to make metrics understandable and actionable, breaking it down into different segments based on business needs. Finally, they need to identify where they need to optimize by thinking about the future state of customer retention, conversion, and satisfaction. By following these steps, companies can improve digital customer experience using observability practices.

Enhancing digital customer experience involves implementing effective observability practices to ensure that applications and services are available, fast, and user-friendly. Companies should focus on measuring key performance indicators (KPIs) such as availability, largest contentful paint, first input delay, and cumulative layout shift to form a quality foundation for customer experience observability. Organizing data by segmenting it based on location, device type, or line of business can make these metrics actionable. By identifying areas needing optimization, businesses can improve customer retention and satisfaction. The article suggests using resources like the New Relic Instant Observability Customer Experience Quality Foundation and the Improve Your Website's Performance tutorial to establish benchmarks and rectify issues like JavaScript errors. Neil MacGowan, with extensive experience in IT and business software, emphasizes that the views in the blog are personal and not necessarily reflective of New Relic's official stance.