March 2022 Summaries

The introduction of open banking regulations in Europe through PSD2 requires banks to enable third-party providers to integrate customer account information and payment capabilities with customer consent. This has led to a significant shift in the way banks approach IT modernization, with many adopting cloud-based architectures and APIs to meet regulatory requirements. Banks that have acted early to implement APIs have seen major advantages, including improved management of customer impacts during COVID-19, wider product offerings, and increased customer engagement. The choice of approach to IT modernization depends on the bank's willingness to move towards a platform strategy, with some adopting architectural transformation strategies and others opting for lift and shift or refactoring existing codebases. Successful modernization requires demonstrating the business case for change and leveraging monitoring and observability tools to show key metrics improvements.

The introduction of the Second Payments Services Directive (PSD2) in 2019 marked a significant shift in banking regulations across Europe, requiring banks to allow third-party access to customer account information and payment capabilities through standardized APIs. While the directive aimed for uniformity, banks often created slightly different APIs, leading fintech companies to integrate with each bank individually. This regulatory change has accelerated the adoption of open banking globally, with 108 countries now following suit, and has prompted banks to modernize their IT infrastructure. Banks have pursued various strategies, such as lift and shift, refactoring, and middleware solutions, to update their legacy systems and meet digital demands. These efforts have enabled them to manage customer impacts of COVID-19, expand product offerings, reach new customer segments, and comply with emerging regulatory requirements. The modernization often involves adopting cloud and API capabilities, with the goal of building a microservices-based architecture that enhances performance and observability. Monitoring API-enabled architectures is crucial for banks to capitalize on the estimated $25 billion API market, focusing on metrics like API adoption rates, usage, change requests, and uptime to improve customer experience and engagement.

Netlify is a web developer platform that offers pre-rendering static content and decoupling of services to improve performance and productivity for developers. However, it lacks built-in traffic logs and function logs storage, requiring monitoring tools like New Relic to store and analyze data. With Netlify Log Drains connected to New Relic, users can capture and analyze web traffic and serverless function logs, monitor user activity and journeys, identify traffic from bots and bad actors, and access long-term log retention for up to 30 days. This setup provides a powerful workflow for optimizing Netlify applications, fixing issues proactively, and gaining insights into user behavior and performance metrics.

Netlify is a web developer platform that enhances application performance by pre-rendering static content and decoupling services, but it lacks built-in long-term traffic log access and function log retention. To address this, users can utilize New Relic, a monitoring tool that connects to Netlify Log Drains, enabling the capture, analysis, and storage of web traffic and serverless function logs. This integration allows for detailed monitoring, visualization, and alerting based on performance data, aiding in application optimization and proactive issue resolution. New Relic offers long-term storage, user traffic monitoring, and user agent analytics, enriching the insights into user activity and security. While Netlify Enterprise accounts provide limited log storage, New Relic extends this capacity and offers an automated dashboard for visualizing key telemetry, thus facilitating a comprehensive log management solution for enhanced observability and application performance.

As a seasoned software developer and conference attendee, I'm excited to return to LeadDev NYC after two years of remote events due to the global pandemic. I look forward to being back in New York City, reconnecting with colleagues, trying new kosher cuisine options, and participating in leadership workshops and speaking sessions on topics such as microservice architectures, software estimation, junior developer support, and effective on-call systems. The event promises to be informative and engaging, with a focus on practical advice for developers at all levels. I'm particularly eager to hear about equipping teams to support junior developers, a topic that resonates with my own experiences as a 54-year-old junior dev in the IT industry.

Leon Adato, a Developer Relations Engineer at New Relic, shares his anticipation for attending the LeadDev NYC event from April 5-7, following a hiatus from in-person events due to the global pandemic. He reflects on his excitement about returning to New York City, reconnecting with colleagues, and exploring diverse kosher cuisine. The event is packed with workshops and sessions aimed at various levels of practitioners, covering topics like observability in microservice architectures, software estimation, supporting junior developers, and improving on-call systems, with a notable session on Capuchin monkeys and team management. Adato, a self-described junior developer at 54, highlights the significance of structuring teams to foster the growth of new developers, emphasizing that being "junior" is a temporary phase. He invites attendees to visit the New Relic booth for demos and discussions, while also noting the range of his extensive IT experience and the views expressed being his own, separate from New Relic's official stance.

APM services are tools that help understand how well an app is performing for customers and end-users, tracking key information like latency, and providing insights to achieve specific SLAs and SLOs. Without APM, businesses risk site latency and downtime, losing customers, sales, and revenue, as well as wasting engineering team time on custom solutions. APM services should include visualization of data, troubleshooting information at the code level, automated service level management, and distributed tracing observability. When evaluating APM tools, consider factors like easy installation, automation, context, alerting integration with existing tools, and AI-powered unified pattern recognition. APM plays a pivotal role in today's software landscape, helping gain deep insights into application behavior, optimizing performance, and ensuring a top-notch user experience.

Join me and the rest of the developer relations team at The Cosmopolitan of Las Vegas on May 17–19 for FutureStack, a software engineering conference that includes a 24-hour hackathon called FutureHack. Developers of any experience level are welcome to participate in the hackathon, which offers prizes in categories such as Continuous Innovation, Build Something Amazing, and Community Hack. The event also features special workshops with partners like Netlify and GitHub, and an opportunity to connect with their engineering teams. A virtual Q&A session is available for those who want insider tips on how to prepare for FutureHack and level up their chance to win the $10,000 prize.

FutureStack, a premier software engineering conference, will take place at The Cosmopolitan of Las Vegas from May 17–19, featuring FutureHack, a 24-hour hackathon open to developers of all experience levels. Participants can engage in various tracks such as Continuous Innovation, Build Something Amazing, and Community Hack, each offering unique challenges and the chance to win substantial prizes, including a $10,000 award for the best overall hack. The event, supported by partners like Netlify and GitHub, provides an opportunity to utilize New Relic technology, with workshops and one-on-one sessions available to enhance project development. Aspiring and experienced developers alike are encouraged to join, with a virtual Q&A session on May 11 offering additional insight and team-building opportunities ahead of the event.

Real user monitoring (RUM) tracks and measures the end user experience in an application, providing valuable insights into how users interact with websites and applications, including page load times, errors, and user satisfaction. RUM is a core feature of application performance monitoring (APM) and provides a more user-centric perspective than server-centric metrics. By implementing RUM, organizations can improve the user experience, detect and address performance bottlenecks, optimize resources, and make data-driven decisions to enhance digital strategy and experience. RUM benefits include improved user experience, faster issue resolution, enhanced conversion rates, data-driven decision making, proactive performance optimization, tailored content and features, and cost optimization. To set up RUM, organizations can implement instrumentation with a monitoring solution such as New Relic, which tracks client-side performance metrics, including timing, golden signals, transaction times, user satisfaction, single-page application (SPA) performance, and JavaScript errors. By following best practices, such as defining clear objectives, prioritizing key user journeys, implementing thoughtful instrumentation, regularly reviewing and analyzing data, monitoring third-party services, integrating with APM and logging solutions, and getting started with a real user monitoring solution, organizations can maximize the effectiveness of RUM and improve their digital platforms' performance.

The practice of Site Reliability Engineering (SRE) applies software engineering skills and principles to improve operational efficiency, data-driven roadmap planning, and general uptime and reliability. SRE is a proactive approach to building and maintaining software that balances innovation with reliability and performance. It involves organizing teams differently depending on the organization and services they support, and it often coexists with DevOps practices. SRE teams are responsible for defining what performant and reliable mean in applications, services, and infrastructure, and they work closely with development and operations teams to understand critical components of their applications and infrastructure. The practice of SRE embraces experimentation, transparency, autonomy, and service levels, which include service level indicators (SLIs), service level objectives (SLOs), and service level agreements (SLAs). Adopting SRE best practices takes time and effort but leads to improved system performance and reliability, benefiting both DevOps teams and customers.

Site Reliability Engineering (SRE) is a practice that combines software engineering principles with operations to improve system reliability and performance, often involving the development of internal applications to address concerns proactively. Despite being popularized by Google in 2016, SRE has been a practice for many years, serving as a bridge between development and operations, particularly within mature DevOps environments. SREs can be integrated into development teams or function as standalone units, focusing on tasks like monitoring, defining reliability metrics, and managing incidents to enhance system performance. SRE principles emphasize balancing innovation with reliability, encouraging collaboration across teams to set service levels and proactively address risks. The adoption of SRE can be tracked through an organization's maturity model, which involves dedicated SRE roles, integration across teams, and autonomy in automating workflows. By utilizing techniques such as continuous integration, monitoring, and service level management, SREs aim to provide transparency and improve the overall customer experience while leveraging modern IT capabilities like automation and AI.

FutureStack22 is a user conference for engineers and developers where they can learn about data-driven observability to open up new possibilities for their work. The event will feature stellar keynotes, hands-on labs, technical deep-dives, and live entertainment on topics such as containerization, cloud migration, machine learning, serverless apps, and more. Attendees can choose from three session tracks: Operate, Shift Left, and Expand to elevate their observability game. Additionally, the conference will include a popular 24-hour hackathon called FutureHack with fun prizes and awards for top projects. Registration is now open with limited spots available for Super Early Bird tickets ending April 15.

FutureStack22, hosted by New Relic at the Cosmopolitan of Las Vegas from May 17-19, is a premier conference for software engineers and developers focusing on data-driven observability. The event offers a platform for attendees to explore advancements in containerization, cloud migration, machine learning, and serverless applications through keynotes, hands-on labs, and technical deep-dives. Participants can choose from three session tracks—Operate, Shift Left, and Expand—each designed to enhance their observability skills and strategies across different stages of production. The conference also features FutureHack, a 24-hour hackathon with prizes for top projects, and provides the opportunity to engage directly with New Relic engineers and be among the first to discover new innovations.

This tutorial explains how to automate common DevOps tasks using Python and the NerdGraph API in New Relic. It covers creating, reading, updating, and deleting tags to segment applications based on environments or other criteria, making it easier to manage and analyze data. The tutorial requires a New Relic account, instrumented application, and API key, and provides code snippets in Python using the requests and json modules. The guide also includes examples of how to use the NerdGraph API to perform these tasks, including error handling and response checking.

New Relic helps Battlesnake players monitor their snake's performance, server, and web application in real-time, allowing them to improve its strategy and make it more competitive. To do this, users need to create a New Relic account, instrument their Battlesnake with custom code, and then analyze the data using dashboards that provide insights into the snake's performance, survival rate, growth, and server status. These dashboards can be customized to show more or less information and are useful for tracking response time, improving code, and gaining a competitive edge in battles. By using New Relic, users can gain a better understanding of their Battlesnake's behavior and make data-driven decisions to improve its performance.

Battlesnake enthusiasts can enhance the performance of their digital snakes by using New Relic to monitor and analyze their Battlesnake's performance, server, and web application in real-time. The process involves creating a New Relic account and instrumenting the Battlesnake using Node.js or other programming languages supported by New Relic's extensive documentation. By setting up New Relic, users can create customizable dashboards that provide insights into their snake's performance, including metrics like competition status, survival rate, and server status. These insights enable users to optimize their snake's response times, track progress, and develop strategies against opponents. Additionally, New Relic offers various integrations and encourages users to share their projects on social media, while highlighting that the views expressed are those of the author and not the company.

In the second part of a five-part series on automating DevOps tasks in New Relic, the blog post explains how to utilize the NerdGraph API to efficiently manage tags for New Relic entities using Python, demonstrating the creation, reading, updating, and deletion of tags through GraphQL queries. The post emphasizes the use of tags for better organization when handling large data sets and offers Python code snippets for each operation, requiring prerequisites such as Python 3, the requests and json modules, a New Relic account and API key, and an application instrumented with New Relic APM and Infrastructure. Readers are encouraged to explore further capabilities of the NerdGraph API for enhancing their observability practices and are invited to engage with the New Relic community via the Explorers Hub for support and discussion.

DAZN, a multi-billion dollar sports subscription platform, has implemented various strategies to manage its complex tech stack and achieve 5,000 live releases per day. The company's "reverse pyramid model" allows teams to choose their own products and monitoring tools, adding to the complexity. To address this, DAZN focuses on testing, retrospectives, and standard documentation flow to identify what went well and what didn't. Craig Mclean, Head of SRE, emphasizes the importance of breaking down big problems into manageable chunks, using microservices, and picking small services with limited impact for initial releases. He also stresses the need to measure key metrics, such as performance issues, and generate stress for testing services to ensure they work under stress. By adopting these strategies, DAZN has been able to achieve its goal of 5,000 live releases per day.

DAZN, a sports subscription platform, navigates the complexity of its multiple engineering teams and diverse tech stack through a "reverse pyramid model," allowing teams to choose their tools while maintaining a standard documentation flow for retrospectives and learning from failures. Craig Mclean, Head of SRE at DAZN, emphasizes the importance of embracing Agile practices and microservices to manage large-scale operations, enabling the company to achieve 5,000 live releases per day. Stress testing and observability are crucial elements of DAZN's strategy to handle high traffic and ensure system reliability, showing where failures occur so they can be addressed efficiently. Mclean advises starting with small, manageable tasks to build confidence in instant releases and focusing on key performance metrics to improve the tech stack over time.

New Relic's NerdGraph API allows users to automate common tasks such as auditing services and exporting telemetry data through API calls in Python or other programming languages. The API uses GraphQL, an open-source format that enables users to request specific data without over-fetching or under-fetching. By using NerdGraph, users can streamline their workflows and improve the efficiency of their observability practices, including automating tasks such as searching for instrumented services and exporting dashboards as files.

The blog post introduces the automation of common tasks in New Relic using the NerdGraph API with Python, focusing on auditing instrumented services as part of a five-part series. It highlights the importance of automation in streamlining DevOps workflows and observability practices, explaining how API calls can be used to audit services efficiently. The post provides a step-by-step guide for using Python to make GraphQL API calls to New Relic's NerdGraph, allowing users to request specific data without over-fetching. It emphasizes the utility of the NerdGraph API for managing entities, exporting telemetry data, and performing other tasks, with each instrumented service identified by a unique GUID. The blog, authored by Steve Ng, also touches on the prerequisites for following the tutorial, including having Python and a New Relic account, and encourages readers to continue with the series to learn more automation techniques.

Cloudflare has integrated its Logpush product with New Relic One, allowing developers to easily monitor and analyze web traffic metrics in a single dashboard. This integration provides visibility into logs and analytics, enabling teams to debug, identify configuration adjustments, and create custom analytics. With the Cloudflare quickstart, users can quickly set up the integration and start seeing key metrics such as saved bandwidth, alerts on cyberattacks, and ranking for fastest and slowest loading pages. The integration eliminates cloud storage middleware costs and provides a seamless process for monitoring web traffic, making it easier to provide an excellent user experience and keep security threats at bay.

Cloudflare and New Relic have launched a new direct integration between Cloudflare Logpush and New Relic One, streamlining the process of delivering log data without the need for middleware, thereby reducing cloud storage costs and enhancing log management efficiency. This integration provides development teams with easy access to key web traffic metrics, such as bandwidth savings, cyberattack alerts, and page load speeds, via a pre-curated dashboard in New Relic One. The partnership aims to simplify the management of diverse applications and infrastructures by offering a seamless log and analytics visibility process, making it easier for teams to correlate data across systems and improve user experience while maintaining security.

APM and observability are two related but distinct approaches to understanding and improving software performance. APM provides users with dashboards and alerts to troubleshoot application performance in production, focusing on known or expected system failures. Observability, on the other hand, is a holistic practice that involves collecting data across multiple layers of software architecture, analyzing it in real-time, and asking questions about that data to understand complex systems and discover unknown issues. While APM provides aggregated metrics, observability platforms give teams a connected, real-time view of their operational data in one place, enabling them to better understand system behavior and make iterative improvements to the entire stack. Observability encompasses multiple data types beyond just metrics, including logs, events, and traces, and aims to enhance the overall understanding of system behavior.

Join Relics like myself and Robert Prast at SRECon in San Francisco from March 14 to 16, where we'll discuss site reliability, systems engineering, and distributed systems at scale. Our booth will feature self-care items for SREs, while we showcase our technology designed to help alleviate their workload. On March 15, attend our talk on triaging security threats with Pixie, an eBPF-powered Observability solution that provides a single source of truth for analyzing data and pinpointing the root cause of attacks. To learn more about Pixie's capabilities, read our blog or visit our documentation, and sign up for a free account with New Relic One to access its features, including auto-telemetry with Pixie.

Join Relics and Robert Prast at SRECon in San Francisco from March 14 to 16 for discussions on site reliability, systems engineering, and distributed systems at scale, with a booth themed around self-care for SREs. The event will include a talk on triaging real-time security threats using eBPF-powered observability with Pixie and New Relic One, offering insights into extracting observability data and analyzing it to identify attack root causes. Attendees can learn more about Pixie's integration capabilities, sign up for a free New Relic One account, and explore documentation for installing auto-telemetry with Pixie. Daniel Kim, the author and Principal Developer Relations Engineer at New Relic, emphasizes the importance of inclusive and accessible developer education and invites readers to discuss the blog post further at the Explorers Hub.

Modern applications often employ a complex architecture of microservices, serverless technologies, and containers to ensure seamless customer experiences, but this can lead to tangled knots when problems arise. To address performance issues quickly, observability practices are necessary to identify the root cause of problems. While log monitoring is widely used, it may not provide enough insight on its own, particularly in complex or hybrid environments where setting up tools requires time and resources that could be spent elsewhere. Instrumentation, which involves adding code to track the inner state of an application, provides broader information about service operations and behavior, including metrics, events, logs, and traces. This approach offers immediate visibility into an application's performance, allows for quick problem-solving, and enables monitoring of complex environments. Relying solely on log monitoring is not recommended, as it may lead to missed insights and delayed troubleshooting.

New Relic One has introduced metric aggregate pruning, a tool to manage dimensional metric cardinality, allowing users to create rules that prioritize important attributes for longer-term trending and analysis while excluding less relevant ones for troubleshooting short-term issues. This feature helps avoid hitting cardinality limits, which can reduce data usefulness. Cardinality refers to the number of unique combinations of key-value pairs in metrics, with high cardinality values requiring more correlated data to analyze. New Relic One uses raw data points and rollups to store and query metrics efficiently. The new pruning ability is ideal for high cardinality attributes like container IDs, which lose value over time but are crucial during incidents. Users can prune attributes using the DROP_ATTRIBUTES_FROM_METRIC_AGGREGATES command or explore other solutions like partitioning data or dropping low-value metrics.

New Relic has introduced metric aggregate pruning, a tool that manages dimensional metric cardinality by allowing users to prioritize certain metric attributes for long-term analysis while deeming others as only relevant for short-term troubleshooting. This feature helps prevent reaching cardinality limits that could hamper data utility by "pruning" unnecessary attributes from queries over extended time spans. The metric aggregate pruning is part of New Relic's data management API, NerdGraph, and is designed to enhance the efficiency of data aggregation by excluding high cardinality attributes, such as container IDs, from metric rollups. This ensures that while raw data remains available, especially for troubleshooting purposes, the system remains performant by avoiding the storage of excessive data. Additionally, New Relic offers solutions like data partitioning and permanently dropping low-value metrics to manage cardinality limits effectively. Users can access detailed documentation and examples to implement these features, with the overarching goal of enhancing observability and data management within their systems.

New Relic has updated its Kubernetes experience to provide a single user interface for monitoring application and cluster performance data, enabling developers to identify and resolve issues faster and scale infrastructure resources as needed. The new experience integrates APM and Kubernetes cluster performance data, providing real-time activity streams, one-click logs, and side-by-side metrics for correlation and investigation. This integration allows developers to understand how Kubernetes infrastructure affects their applications and troubleshoot or optimize performance more effectively. With the updated experience, users can investigate anomalies faster by accessing relevant logs and correlating performance across metrics using pre-built queries. The new features are now available to all full platform users, with a free account option including 100 GB/month of data ingest and unlimited basic users.

The New Relic Instant Observability (I/O) platform offers a free and open ecosystem of over 750 quickstarts that enable developers to start monitoring their applications in minutes without manual setup. These quickstarts bundle necessary building blocks for instrumenting, monitoring, and acting on signals from telemetry data, covering popular technologies such as Java, .NET, and Laravel. New Relic I/O has recently expanded its partner integrations with leading enterprise software partners, including CircleCI, Netlify, ReleaseIQ, Mule ESB, Databricks, and Atlassian Jira for Errors Inbox, providing instant observability for CI/CD platforms, integration platforms, data analytics, and agile project management. The new quickstarts offer features such as integrating New Relic One with popular cloud services, tools, and open standards, allowing developers to visualize, monitor, and act on all their data in a single platform.

New Relic Instant Observability (I/O) is an open ecosystem offering over 750 free, pre-built quickstarts designed to facilitate rapid monitoring of technology stacks without manual setup. These quickstarts cover a range of technologies, including Java, .NET, and Laravel, and include integrations from enterprise software partners, enabling users to easily visualize, monitor, and act on data from a single platform. The latest integrations include partners like CircleCI for CI/CD, Netlify for web application monitoring, ReleaseIQ for software delivery, Mule ESB for integration platforms, Databricks for data analytics, and Atlassian Jira for agile project management. These partnerships provide detailed dashboards, alerts, and insights, enhancing the observability of various systems and helping teams improve efficiency and performance. The platform also connects New Relic One observability data with tools like Jira to streamline error tracking and resolution, making observability a habitual practice for engineers.