November 2019 Summaries

The New Relic One observability platform has become programmable, allowing users to connect their observability data to business decisions. To showcase this capability, New Relic has developed a handful of open-source applications that can be deployed on the platform. These applications include Container Explorer, Deployment Analyzer, and Page View Map, which provide tools for visualizing container inventory, deployment analysis, and geographic visualization of browser application data. The applications are available under an open-source license and can be deployed locally using the New Relic CLI or published to a personal account. New Relic also offers resources for building custom applications, including the Developer Edition and SDKs, as well as community support through the Explorers Hub. Contributions to these open-source projects are welcome, and users can preview more apps in future editions of this series.

New Relic One's observability platform has become programmable, allowing users to build custom applications that connect observability data to their business needs using the same tools available to New Relic engineers. The platform supports the creation and deployment of open-source applications such as Container Explorer, Deployment Analyzer, and Page View Map, which facilitate container management, application deployment visualization, and geographic data mapping, respectively. These apps are available on New Relic's GitHub repository and can be deployed with the necessary New Relic tools and subscriptions. Additionally, New Relic offers resources like the Developer Edition, GraphQL Explorer, and extensive documentation to support users in developing their own applications. The platform encourages community contributions to its open-source projects, with guidelines and support provided for potential contributors.

New Relic's commitment to removing barriers that keep young people from pursuing careers in tech is a pledge that makes sense on multiple levels: as a solution to the industry's talent crunch; as a tactic for promoting opportunity and inclusion within communities that have often been under-represented in the tech field; and as an issue that many Relics feel strongly about. Tech Trips, which were supported by New Relic's first Global Impact Campaign, aimed to bring together students with Relics from various backgrounds to connect them with powerful role models for their own career journeys. The campaign was truly global, with events held in eight offices across the world, involving over 100 Relics and more than 240 students worldwide. Many of these events were hands-on activities that highlighted problem-solving skills, teamwork, and career development opportunities. New Relic's leadership team is passionate about innovation and diversity, and the company believes that building a more diverse and inclusive tech industry is crucial for everyone's success. By focusing on global teams serving local communities, New Relic aims to make technology accessible and enablement for nonprofits.

New Relic's Tech Trip initiative aims to dismantle barriers preventing young people from pursuing careers in technology by partnering with local charities and schools globally, highlighting their commitment to diversity and inclusion in the tech industry. Through the NewRelic.org Global Impact Campaign, over 100 employees engaged with 240 students across eight offices, providing hands-on activities, career advice, and relatable role models to inspire students from underrepresented communities. These efforts were part of a broader strategy to foster a more diverse tech workforce by leveraging New Relic’s global presence and resources to make a local impact. Events were held in multiple cities, including Barcelona, Dublin, Atlanta, Phoenix, San Francisco, Portland, Sydney, and Tokyo, each tailored to the specific needs of the communities, with activities ranging from panel discussions and resume workshops to school visits and interactive problem-solving exercises. The initiative underscores New Relic's dedication to social impact and the belief that everyone stands to benefit from a more inclusive tech industry, supported by a leadership team passionate about innovation and technology.

New Relic will be attending AWS re:Invent 2024, a four-day conference in Las Vegas, featuring talks, bootcamps, labs, keynotes, and parties. New Relic experts will provide lightning talks and expert consultations at booth #401 in the Venetian Expo Hall, covering topics such as modernizing software delivery, cloud migration, performance optimization, monitoring application performance, Kubernetes, infrastructure, serverless applications, and more. The company will also host 1:1 consultations with its technical team to discuss various topics related to New Relic One, its observability platform. Additionally, New Relic will be exhibiting at the AWS Partner Solutions Experience and hosting a breakout session on top container and Kubernetes best practices. The event also includes networking opportunities, such as receptions and a client appreciation evening, as well as a Serverless for Everyone Community Party.

AWS re:Invent is set to take place from December 2, with New Relic actively participating by showcasing its observability platform, New Relic One, at their booth in the Venetian Expo Hall. New Relic will deliver lightning talks and offer expert consultations on topics such as software modernization, cloud migration, performance optimization, and serverless monitoring, leveraging their recent acquisition of IOPipe. Attendees can engage in hands-on activities like building applications in the Hacker Lounge or participating in AWS GameDay, which tests AWS skills through a gamified simulation exercise. New Relic will also host breakout sessions on container best practices and building microservices with AWS services. Networking events, including receptions and community parties, will provide opportunities to connect with industry peers and discuss the latest advancements in software development and cloud services.

Prometheus is an open-source toolkit that provides monitoring and alerting for services and applications running in containers, widely used to collect metrics data from Kubernetes environments. The New Relic Prometheus OpenMetrics integration allows users to send their Prometheus metrics directly to New Relic for storage, visualization, and correlation with other telemetry data, removing the overhead of managing Prometheus storage and availability. With this integration, users can monitor key aspects of their Kubernetes environments, such as etcd performance and health metrics, Kubernetes horizontal pod autoscaler (HPA) capacity, and node readiness. The integration supports both Docker and Kubernetes, using Prometheus version 2, and provides tools to visualize and alert on these metrics, making it easy for organizations to get started with Prometheus and scale their data without the hassles of managing Prometheus and a dashboard tool like Grafana.

The text discusses the integration of Prometheus, an open-source monitoring tool widely used in Kubernetes environments, with New Relic's platform via the Prometheus OpenMetrics integration. This integration allows users to send metrics directly from Prometheus to New Relic for enhanced storage, visualization, and correlation with other telemetry data, simplifying the management and scaling challenges associated with Prometheus. The document outlines the installation process within a Kubernetes cluster and details how users can query and visualize the metrics using New Relic's tools. Additionally, it highlights various use cases for the integration, such as monitoring Kubernetes' etcd, Horizontal Pod Autoscaler (HPA), and node readiness, while emphasizing New Relic's contribution to the open-source community and inviting further collaboration.

AWS customers can now use third-party resource providers in their application stacks configured with CloudFormation, allowing them to create NRQL alert conditions as resources directly in CloudFormation templates, which enables flexible and customizable alerting for monitored services and applications. This allows developers to easily integrate New Relic One into their existing AWS infrastructure using a familiar tool like CloudFormation, streamlining observability practices and improving the overall quality of software development and deployment processes.

The New Relic Logs team has partnered with Amazon Web Services (AWS) to support FireLens custom log routing for Amazon Elastic Container Service (ECS) environments, enabling teams to send ECS log data to New Relic and gain additional visibility into their environment's health and performance. FireLens is a fast, simple, and efficient process that adds a sidecar container using Fluent Bit or Fluentd to function as a log router, allowing for custom log routing within ECS task definitions. Configuring FireLens for New Relic Logs is a two-part process, involving the configuration of the log router container definition and the setup of logging configurations for application containers, which can be done with minimal effort and leveraging AWS Secrets Manager for security. With FireLens up and running, teams can explore their data using the New Relic Logs UI, view contextual log data, query their data in New Relic Insights, and review documentation for detailed setup, configuration, and troubleshooting tips.

Infrastructure-as-code (IaC) is a common practice among DevOps teams for managing cloud applications, and AWS CloudFormation facilitates this by allowing teams to define their infrastructure using JSON or YAML templates. With CloudFormation, users can now integrate third-party resource providers, such as New Relic's NRQL alert conditions, directly into their application stacks. This integration allows for the creation of flexible alerting mechanisms that monitor services and applications for violations, such as unexpected increases in resource usage. The process involves using CloudFormation templates to define AWS resources and interactions, which are then built by CloudFormation's execution engine through API calls. New Relic offers a solution through its newrelic-alerts-cloudformation project, enabling the creation or updating of NRQL alerts from CloudFormation stacks. This approach supports New Relic's emphasis on observability as key to developing robust software, providing tools for monitoring and responding to infrastructure changes effectively.

New Relic has partnered with Amazon Web Services to enhance log data management for Amazon Elastic Container Service (ECS) environments through the integration of FireLens, a tool that enables custom log routing within ECS task definitions. This collaboration allows teams to efficiently send ECS log data to New Relic, providing increased visibility into ECS environment performance. FireLens uses a sidecar container with Fluent Bit or Fluentd as log routers, simplifying the process compared to other options like AWS CloudWatch. The setup process involves configuring FireLens log router container definitions and logging configurations for application containers, with New Relic offering custom images for various AWS regions. Once configured, the integration immediately starts generating log data, which can be explored using New Relic's tools, offering insights into container performance and enabling advanced log management capabilities.

New Relic and Google Cloud have collaborated closely to help customers accelerate their migration to and adoption of Google Cloud solutions, particularly with robust solution integrations for services including Google Kubernetes Engine (GKE), Google Cloud Anthos, and Cloud Run. New Relic One is the industry's first open, connected, and programmable observability platform that provides data gathering, visualization, and analysis of metrics, events, logs, and traces in one place. The solution integrations for Google Cloud Anthos and Cloud Run expand New Relic's portfolio for Google Cloud, enabling customers to monitor frontend and backend applications as well as the underlying hosts running their clusters. With New Relic APM 360, customers can instrument to monitor applications running in containers and correlate data across the entire stack, while New Relic Distributed Tracing provides a curated dashboard experience that abstracts and demystifies the complexity of the Kubernetes and serverless deployment stack. The solution also includes logging information across multiple layers, providing correlation and context with metrics, events, and tracing data coming out of the stack.

New Relic and Google Cloud have expanded their collaboration to enhance solutions for customers adopting Google Cloud services, particularly Google Cloud Anthos and Cloud Run. Anthos is a hybrid and multi-cloud platform that enables the management of applications across diverse environments using Kubernetes, while Cloud Run offers serverless deployments for containers. New Relic One, an open and programmable observability platform, integrates with Anthos and Cloud Run to provide comprehensive monitoring, visualization, and analysis of metrics, logs, and traces. This collaboration offers infrastructure-centric and application-centric views, enabling businesses to monitor Kubernetes clusters, containers, and applications effectively. The integration includes tools like Kubernetes cluster explorer and New Relic Distributed Tracing, which help diagnose performance issues and improve application visibility. Additionally, New Relic's capabilities for logging, event monitoring, and Istio metrics ensure detailed insights into deployments, aiding businesses in optimizing their cloud infrastructure and application performance.

New Relic has introduced two new ways to add monitoring and observability to AWS Lambda functions without requiring code changes, including a Serverless Plugin for organizations using the Serverless Framework and a manual installation option that reduces onboarding friction. These updates aim to simplify serverless monitoring by providing real-time high-resolution invocation metrics and intuitive debugging tools. New Relic also announced the addition of the IOPipe team to its serverless team, which will focus on integrating key technologies into the New Relic One observability platform.

Earlier this year, New Relic introduced New Relic Monitoring for AWS Lambda as part of its observability platform, New Relic One, which was well-received by customers who appreciated the enhanced visibility into AWS Lambda-based applications. The company has since streamlined serverless function monitoring by launching two new methods for adding monitoring and observability to serverless functions without code changes. These include a Serverless Plugin for real-time metrics and a manual addition of New Relic’s AWS Lambda layer for simpler instrumentation. Lambda layers, which support Node.js and Python, allow users to include libraries in their functions without adding them to the deployment package, reducing errors and package size. New Relic has also enhanced its serverless team by integrating the IOpipe team, which will contribute to the development and integration of AWS Lambda Layers and deployment frameworks into the New Relic One platform. The company encourages users to explore these innovations at AWS re:Invent 2019 and offers a free trial to get started with their serverless monitoring solutions.

Fluent Bit is a log collector that can be used with New Relic Logs for Kubernetes to collect cluster log data, and it allows developers to control which logs are collected by using two different strategies: editing the parser configuration or using Kubernetes annotations. The second strategy, using annotations, provides more flexibility and decentralization of the logging configuration, allowing developers to adjust their logging needs as they add new pods to a cluster. By enabling New Relic Logs for Kubernetes and choosing one of these strategies, developers can get what they need from the logs while also optimizing costs by not sending unnecessary data.

Log data is essential for troubleshooting in complex platforms like Kubernetes, and New Relic provides a Fluent Bit output plugin that can be deployed as a Helm chart to collect cluster log data. A customer inquired about disabling certain logs, such as NGINX access logs and Node.js logs, to optimize costs and avoid unnecessary data collection. Two strategies were discussed for managing which logs Fluent Bit should collect: editing the parser configuration or using Kubernetes annotations. The first strategy involves modifying the parser configuration, which centralizes changes but can complicate management if new pods or logs are needed. The second strategy, using Kubernetes annotations, offers more flexibility by allowing developers to control logging on a per-pod basis, requiring the enabling of the K8S-Logging.Exclude feature in Fluent Bit. The blog suggests that annotations provide a better solution by decentralizing log configuration, thus accommodating dynamic changes in a Kubernetes environment.

The Ruby programming language was created by Yukihiro "Matz" Matsumoto in 1995 and first released as version 0.95. It gained popularity with the release of David Heinemeier Hanssen's Ruby on Rails web application framework in 2004. Today, Ruby is used globally for various applications, including NASA simulations, security frameworks, and web development. The language has a large online community known as "Rubyists" who contribute to its growth and evolution. The list includes notable developers who have played a significant role in Ruby's development, such as Sarah Allen, Avdi Grimm, Aman Gupta, David Heinemeier Hanssen, Terence Lee, Linda Liukas, Akira Matsuda, Yukihiro "Matz" Matsumoto, Desi McAdam, Sarah Mei, Sandi Metz, Erik Berlin, Charles Nutter, Katrina Owen, Aaron Patterson, Mike Perham, Evan Phoenix, Sam Saffron, Koichi Sasada, Charlie Somerville, and many others. These developers are recognized for their contributions to Ruby's growth, their expertise in the language, and their advocacy for coding education and diversity.

Ruby, an influential programming language created by Yukihiro "Matz" Matsumoto in 1995, gained significant traction with the introduction of Ruby on Rails by David Heinemeier Hanssen a decade later. The language has since developed a vast global community of "Rubyists" who utilize it for diverse applications, from NASA simulations to web applications like Basecamp. Prominent figures in the Ruby community include Sarah Allen, who founded Bridge Foundry and co-founded RailsBridge to promote coding among underrepresented groups, and Avdi Grimm, a renowned Ruby consultant and trainer. Other notable contributors include Aman Gupta, a Ruby core committer; Terence Lee, a key figure at Heroku; Linda Liukas, who advocates for early coding education; and Akira Matsuda, a leading Ruby community organizer in Japan. The community also highlights influential developers like Desi McAdam, Sarah Mei, Sandi Metz, and others whose work in Ruby has been pivotal in evolving the language and its applications.

New Relic One has introduced programmability to its observability platform, allowing users to connect their observability data to their business and make data-driven decisions. To demonstrate this, New Relic has created several open-source applications, including Status Page, GitHub Integration, Agent Groundskeeper, and Customer Journey, which can be deployed locally or in a New Relic account. These apps provide various features such as aggregating system status, integrating with GitHub, displaying real-time version information, and examining customer journeys. Users can deploy these apps using the New Relic CLI and require access to New Relic One, a personal API key, and Node v10 or higher. New Relic also provides resources for building custom applications, including the Developer Edition, GraphQL Explorer, and NR1 Workshop. The company welcomes contributions to its open-source projects and encourages users to join the Explorers Hub for support and discussion.

New Relic One has made its observability platform programmable, allowing users to build custom applications that integrate various observability data sources into a centralized location, enabling data-driven decision-making. The blog highlights several open-source applications that can be deployed, including Status Pages, GitHub Integration, Agent Groundskeeper, and Customer Journey, each offering distinct functionalities such as system status monitoring, repository documentation access, real-time version information, and user journey analysis. These applications are designed to enhance the user experience by providing quick access to critical information and facilitating efficient data management. Additionally, New Relic encourages community contributions to these projects, provides resources for building custom applications, and emphasizes the importance of open-source collaboration in expanding the platform's capabilities.

The New Relic One observability platform has adopted open instrumentation, allowing it to collect telemetry data from both open and proprietary sources. The company has built an adapter for Istio, a popular service mesh, to integrate with its telemetry engine, Mixer, and send curated telemetry data to the platform. This adapter uses the go-telemetry-sdk to gather metric and trace data, including Istio's metric and trace telemetry, and sends it to New Relic as open source dimensional metrics. The adapter can be used by any service running in the Istio service mesh without requiring reconfiguration or a New Relic agent. The adapter is now available on GitHub and has been open sourced as part of New Relic's initiative to promote open instrumentation.

New Relic has developed the newrelic-istio-adapter, an open-source tool designed to integrate Istio's telemetry engine, Mixer, with the New Relic observability platform, allowing for the collection and analysis of telemetry data from the Istio service mesh in Kubernetes environments. This adapter leverages the go-telemetry-sdk to transmit both metric and trace data to New Relic, enabling users to monitor critical service metrics such as error rates, latency, request volume, and throughput without the need for additional agents. The initiative aligns with New Relic's commitment to open instrumentation, providing a means to send curated multidimensional metrics and distributed tracing spans to New Relic, enhancing observability for services within the Istio service mesh. The adapter, available on GitHub, offers configurations for easy deployment and integration, and contributions to its development are welcome, with detailed instructions and resources provided for users and developers.

New Relic has always focused on helping developers build better software faster, and now they're offering a free Developer Edition with up to $500 worth of product usage per month for non-production workloads. This allows developers to explore the full power of New Relic's observability platform, including its Programmability capabilities, training, and sample code. The company has built a comprehensive platform that makes building better software possible, and they're making it easy for developers to try out all the features with the Developer Edition. With access to the New Relic One CLI, open source apps and code examples, and community support, developers can innovate and experiment with the platform without breaking the bank. The views expressed in this blog post are those of the author and do not reflect the views of New Relic, but provide a unique perspective on the company's mission to help developers build better software.

As an early-career engineer, I sometimes feel overwhelmed by new information and concepts, but I've found a phrase that helps me keep my cool: "I'm not expected to know this yet." This phrase is written on a sticky note attached to my monitor and reminds me that it's okay to not have all the answers. To stay on top of learning and retention, I use various strategies such as keeping a journal of topics to research later, asking questions at the right time, seeking help from more experienced team members, learning at my own pace, identifying opportunities to learn and take advantage of them, and finding a mentor. These strategies have helped me develop a skill to parse information, learn, and retain knowledge while maintaining realistic expectations. By adopting these approaches, early-career engineers can also benefit from managing their expectations and staying positive as they navigate complex topics.

Early-career engineer Caitlin Halla shares her strategies for managing the overwhelming influx of information in her role, emphasizing the importance of recognizing that it's okay not to know everything immediately. She suggests practical approaches such as maintaining a journal for topics to explore later, asking questions strategically, seeking help from experienced colleagues, learning at a comfortable pace, identifying learning opportunities, and finding mentors. These methods help maintain perspective and set realistic expectations, underscoring the necessity of time and patience in developing an engineering career. Halla also highlights the supportive role of team members and mentors in navigating the challenges of a new career.

New Relic has announced the early access release of its free Developer Edition, offering two users up to $500 worth of product usage per month, aimed at supporting non-production workloads across its platform. This initiative is part of New Relic's goal to enhance developer experience by providing easy access to its observability platform, allowing for experimentation and innovation. The Developer Edition includes the same data retention period as the Pro-level subscription and encourages developers to explore and leverage New Relic’s capabilities, supported by training, sample code, and a vibrant community. Announced at Futurestack New York by CEO Lew Cirne, the platform’s programmability is highlighted as a key feature, furthering New Relic's evolution into a comprehensive observability platform. Joel Worrall, Director of Open Source and Developer Advocacy, emphasizes the company's commitment to building tools that help developers create better software, inviting developers to engage through the Explorers Hub for support and discussions.

At New Relic, we believe serverless is a key part of a meaningful shift in how teams build better software. Customers like Morningstar have rebuilt their web presence using AWS Lambda and chosen to use New Relic Serverless for tuning their site and network of dependencies. New Relic has acquired IOpipe, a pioneer in serverless monitoring and observability, to support this growth. IOpipe provides detailed data that allows teams to troubleshoot complex workloads. With the acquisition, New Relic will enhance its New Relic Serverless solution and support other FaaS and serverless technologies. The team behind IOpipe shares a vision with New Relic to help modern software teams build better software. Gartner predicts that 100% of enterprises using IaaS or public cloud will use some serverless platform as a service in production applications by 2023. The acquisition allows New Relic to make investments in onboarding and instrumenting AWS Lambda functions, enabling customers to rapidly build fast and reliable production workloads with the platform.

New Relic is advancing its serverless technology offerings by acquiring IOpipe, a company noted for its expertise in serverless monitoring and observability, to enhance the performance of AWS Lambda functions. This acquisition aims to integrate IOpipe's technology into the New Relic One observability platform, making it easier for customers to monitor and troubleshoot complex serverless applications. With IOpipe's co-founders, Erica Windisch and Adam Johnson, joining New Relic, the company plans to accelerate innovation and support other serverless technologies beyond AWS Lambda. The partnership reflects a shared vision to improve serverless application development, anticipating increased adoption of serverless platforms as predicted by Gartner. The integration is expected to foster better software development and operational efficiency for a diverse range of customers, including well-known companies such as Morningstar and Comic Relief.