May 2021 Summaries

This paragraph provides an objective summary of the text, highlighting key points about Kubernetes and its role in orchestrating and managing application containers. It mentions that Kubernetes has emerged as the de facto standard for container orchestration and management, but is still in its early journey towards achieving its full potential. The paragraph also notes that many customers find value in using New Relic's observability platform to monitor their applications running on Kubernetes, and how this can help them troubleshoot issues more efficiently. Additionally, it explains the importance of monitoring Kubernetes application performance and provides guidance on choosing a tool for monitoring, setting up instrumentation, and linking Kubernetes metadata to application instrumentation. The paragraph concludes by emphasizing the need for comprehensive coverage, timely issue detection, and efficient troubleshooting in implementing effective Kubernetes application monitoring strategies.

Kubernetes is an open-source container orchestration platform designed to automate the deployment, scaling, and management of containerized applications. It has become the defacto standard for container orchestration and is backed by key players like Google, AWS, Microsoft, IBM, Intel, Cisco, and Red Hat. Kubernetes helps organizations move faster, is cost-efficient, cloud-agnostic, and provides a self-service platform-as-a-service that creates a hardware abstraction layer for development teams. It also offers features such as load balancing, security, monitoring, and logging to simplify the management of containerized applications. Despite its benefits, Kubernetes adoption can be challenging due to factors like complexity, skill acquisition, and managing large-scale deployments. To overcome these challenges, it's essential to follow best practices such as resource optimization, secret management, network policies, RBAC, and backup and disaster recovery. Additionally, tools like Helm and Istio can help streamline the management of Kubernetes applications, while New Relic provides a platform for monitoring and analyzing Kubernetes clusters and workloads.

New Relic is expanding its platform capabilities by integrating with Kentik to provide network context, enabling engineers to rule out or rule in the network as the cause of an issue and improve problem-solving efficiency. The partnership provides a unified view of application and infrastructure observability data, allowing teams to identify and troubleshoot issues more effectively. With this integration, New Relic users can add network telemetry data from Kentik, visualizing it alongside application and infrastructure data in a single platform, streamlining issue resolution and reducing latency and downtime.

The company is offering a free student edition of its observability platform, allowing students to gain valuable skills in DevOps and site reliability engineering, while also promoting diversity in tech careers by targeting underrepresented groups. The platform provides 500 GB of data per month, three full-stack users for collaboration, training and certifications, and access via the GitHub Student Developer Pack. This initiative aims to inspire more people from diverse backgrounds to pursue a career in tech, as highlighted by the Accelerate State of DevOps 2019 report.

Kubernetes has become the industry standard for managing application containers, and despite its widespread adoption, it is still evolving, with future environments expected to be vastly more complex and challenging to monitor. New Relic supports Kubernetes by offering enhanced observability through its Auto-telemetry with Pixie feature, which simplifies the process of monitoring Kubernetes clusters and workloads without the need for manual instrumentation or code updates. Kubernetes application monitoring is crucial for organizations to maintain the health and performance of applications in dynamic environments, with New Relic providing solutions that link application performance data to Kubernetes metadata, enabling efficient debugging and troubleshooting. This integration allows users to monitor key performance metrics such as CPU, memory, and network usage, and utilize a user-friendly interface to visualize and manage these metrics. Effective monitoring involves setting clear objectives, collecting comprehensive metrics, and establishing centralized logging and alerts, ensuring applications run reliably and efficiently, ultimately enhancing user experiences.

The New Relic Student Edition is a program designed to help students at all educational levels gain valuable and in-demand tech skills by providing free access to New Relic's observability platform. Aimed at aspiring tech professionals, particularly those interested in fields like DevOps and site reliability engineering, the program offers 500 GB of data per month, collaborative opportunities with three full-stack users, and training and certifications that can be added to resumes. The initiative, also available via the GitHub Student Developer Pack, seeks to increase diversity in tech professions by encouraging participation from underrepresented groups and providing resources often lacking in traditional computer science and coding bootcamp curricula.

Engineers are increasingly reliant on the New Relic platform for efficient software deployment and monitoring, as it integrates network performance telemetry data from Kentik into New Relic One, enhancing problem-solving capabilities. This collaboration enables engineers to visualize data from various development stages in one place, facilitating faster and more effective software development. The partnership addresses the challenges of fragmented monitoring tools by providing a unified observability solution that helps teams quickly identify and troubleshoot application issues related to network performance. Through features like automatic network entity tagging and real-time anomaly detection, New Relic and Kentik's integration allows DevOps and NetOps teams to efficiently manage and diagnose network-related problems without unnecessary disruptions. The partnership's expansion, marked by an early access program, underscores the goal of deeper tool integration in the future to improve the overall user experience and system reliability.

Letgo, a mobile app for buying and selling second-hand goods, faced challenges in managing its error resolution process due to tool sprawl with its previous logging tools. The company migrated from an ELK Stack to New Relic One, which streamlined its troubleshooting process by providing a single, high-level view of its Kubernetes clusters and allowing teams to quickly navigate to specific transactions within apps. With New Relic One, letgo's product team can now focus on developing and scaling their software instead of managing external systems, resulting in faster, simpler troubleshooting in just a few clicks.

Letgo, a mobile app designed for selling second-hand goods, has achieved over 100 million downloads by simplifying local buying and selling. As the app's traffic increased, the company faced challenges in efficiently resolving errors, which initially involved using an ELK Stack that proved cumbersome due to tool sprawl and distributed logging data. To address these issues, Letgo transitioned from EC2 to Kubernetes and adopted New Relic One for a more streamlined observability strategy. This migration allowed their product team to enhance error resolution by offering a consolidated view from high-level Kubernetes clusters to specific transactions, significantly reducing troubleshooting time and allowing developers to focus more on innovation rather than managing external systems.

The New Relic team has updated their Ruby agent for auto-instrumentation to help developers identify and solve performance issues, collect and analyze business data, and make more data-driven decisions. The new version, 7.x series, includes changes such as ending support for Ruby 2.0 and 2.1, removing deprecated APIs, making prepend the default for auto-instrumentation, and providing consistent configuration attributes. To help users transition to the new agent, New Relic offers a certification program, documentation, and migration guides, as well as resources on how to get started with programmability and custom visualizations. The team encourages developers to join the conversation in the Explorer's Hub and subscribe to Nerdlog emails for weekly updates on features and releases.

New Relic has updated its Ruby agent to version 7.0 to improve auto-instrumentation, ending support for Ruby 2.0 and 2.1, and removing deprecated APIs to streamline performance monitoring and enhance data consistency. The update includes new features like the adoption of the W3C Trace Context and Infinite Tracing for distributed tracing, while making prepend the default for auto-instrumentation. New Relic also offers a certification program to help users maximize the potential of New Relic One, encouraging the creation of custom applications and visualizations through its open-source integrations and tools. The company emphasizes its commitment to open observability and offers resources such as documentation and a migration guide to assist users in transitioning to the new Ruby agent version.

The FutureHack at FutureStack is a 24-hour virtual hackathon that offers four different tracks for various experience levels, including coders, non-coders, seasoned professionals, and students. All participants can win great prizes, learn about New Relic One, sharpen their coding skills, and meet new people, including potential future friends or colleagues. The event is designed to help individuals level up their professional skills, collaborate with others, and stage their next breakout moment after a challenging year. Participants can choose from tracks such as Low-Code, No-Code, Build on New Relic, Hack for Good, and Error in Prod, each offering unique opportunities to learn and showcase their skills.

FutureHack at FutureStack is a 24-hour virtual hackathon organized by New Relic, designed for participants of all experience levels, including coders, non-coders, seasoned professionals, and students, offering opportunities to enhance professional skills, collaborate, and win prizes. The hackathon features four tracks: a Low-Code, No-Code track suitable for business analysts and TechOps; a Build on New Relic track for creating custom visualizations; a Hack for Good track focused on community-benefiting projects; and Error in Prod, a student workshop for site reliability engineering. Participants can explore New Relic One, build dashboards, and engage in a scavenger hunt, with categories offering substantial prizes such as $10,000 for the Best Overall Hack and various tech gadgets. The event aims to facilitate networking and skill development as the world reopens, with registration leading to an email for track selection and team formation.

FutureStack 2021, a virtual tech conference sponsored by AWS, is just around the corner and promises to be an exciting event for engineers, developers, and IT leaders. The three-day fully virtual conference features a range of activities, including a hackathon, inspiring keynotes from industry experts, a live music performance by Sonic Pi, and a special awards ceremony celebrating customer innovation and excellence in observability. Attendees can also take part in yoga sessions, watch a live Lego build, and enjoy a variety of mood-enhancing rooms to help them pace themselves throughout the event. With top-notch speakers, engaging tracks, and exciting competitions, FutureStack 2021 is an event not to be missed for anyone interested in software development and innovation.

The author of this post discusses how to handle multiline log messages in New Relic Logs using custom Fluent Bit configurations on Linux and Windows. The author explains that out-of-the-box functionality provided by the Infrastructure agent is limited, particularly when it comes to parsing and displaying multiline log messages. To overcome this limitation, the author creates a custom Fluent Bit configuration and parsers file to decorate the log with the source file name, output the parsed log with the key name, and use a Regex pattern to mark the timestamp, severity level, and message from the multiline input. The author provides a YAML configuration file that can be used on Linux and Windows platforms, noting that these configs have strict indentation requirements and require spaces instead of tabs. With this custom configuration in place, the author is able to see each multiline message displayed as a single entry in New Relic Logs.

The text explains how to handle multiline log messages using New Relic's Infrastructure agent, which includes a Fluent Bit plugin for log forwarding. It outlines the challenges of managing multiline logs and provides a detailed guide on creating a custom Fluent Bit configuration to consolidate these logs into single entries for better visualization and storage in New Relic Logs. This involves using a YAML configuration file and a parsers file to direct Fluent Bit to process multiline logs using specific Regex patterns. The text emphasizes the importance of adhering to strict syntax rules, especially for Windows users, and highlights the role of Zack Mutchler, a Senior Manager in Technical Marketing at New Relic, in developing observability solutions. It also notes that the views expressed are personal and not necessarily those of New Relic, while encouraging readers to engage at the Explorers Hub for further discussion.

FutureStack 2021, a major virtual tech conference hosted by New Relic, is set to take place from May 25 to May 27, offering a comprehensive experience for engineers, developers, and IT professionals. Sponsored by AWS, the event includes a 24-hour hackathon, keynote speeches from notable industry leaders, and unique activities such as a live Lego build by a Lego Masters winner and a special Sonic Pi music performance. Participants can engage in diverse tracks, including charity-focused initiatives, and enjoy various entertainment and relaxation options like the Nerd Island track and mood rooms. The event aims to inspire and connect the global tech community through innovative sessions, networking opportunities, and fun challenges while promoting charitable contributions through swag exchanges. Registration is free, and the full schedule is available on the FutureStack website, providing an opportunity to explore the latest trends in software development and observability.

At New Relic, we are committed to making the lives of every engineer easier by open sourcing our instrumentation and browser real user monitoring agent. We have doubled down on this commitment with the announcement that we are now a Platinum member of the Cloud Native Computing Foundation (CNCF) and joining the governing board, and recently open sourced our Browser agent in GitHub. The new open sourced New Relic Browser agent provides complete transparency about the code deployed to systems, allows better engagement with users, and accelerates innovation and quality of browser monitoring product. The agent offers real user monitoring, tracking user perception, and a full-stack observability approach through distributed tracing, providing insights into page performance and user experience. With the open sourced Browser agent, users can start contributing feedback, increasing innovation and quality, by following a step-by-step guide or directly contributing to the project.

New Relic has reinforced its commitment to open-source development by making its Browser agent for real user monitoring (RUM) open source, reflecting its mission to ease engineers' workloads. As a Platinum member of the Cloud Native Computing Foundation (CNCF), New Relic continues to advance its open-source efforts. The Browser agent, now accessible on GitHub, offers comprehensive insights into application performance by tracking real user sessions, identifying issues like slow page loads and JavaScript errors, and measuring user satisfaction through metrics such as Google's Core Web Vitals. The agent also employs distributed tracing for a full-stack observability approach, enabling users to follow requests from browser activity through the network to backend processes. New Relic encourages contributions from the community through its Open Source site, providing resources like a Contributor’s Guide and a Beginner’s Guide to facilitate participation and innovation.

Bill Staples has been promoted to CEO of New Relic, effective July 1, 2021, after serving as President and Chief Product Officer since February 2020. Bill's vision for the company's strategy and opportunity resonated with the founder, Lew Cirne, who will step down as CEO and become Executive Chairman. Bill's leadership has driven significant transformation, including a shift to a consumption-based business model, major product releases, and acquisitions. He is seen as a strong strategic and operational executive who lives the New Relic core values of being bold, passionate, accountable, authentic, and connected to customers and users. Lew Cirne will support Bill as he leads the company into its future, while Hope Cochran takes on the new title of Vice Chair and Lead Independent Director. The promotion is subject to forward-looking statements and assumptions about New Relic's future growth and market position.

The disconnect between IT operations teams and business stakeholders often leads to tension and limited success. This disconnect arises from different definitions of success, with IT measuring resource consumption and the business measuring speed and quality. To resolve this issue, service-level indicators (SLIs) and service-level objectives (SLOs) can be used. SLIs are metrics that measure aspects of the level of service provided, while SLOs are goals set for these metrics. By focusing on SLIs that measure business-impacting aspects such as latency, error rate, and availability, and presenting them as percentages, IT teams can align with the business's expectations and improve their observability practice.

At New Relic, we are committed to open observability for all, embracing principles of open exchange, collaborative participation, and community-driven development. We've made our docs, Pixie, integrations, SDKs, and instrumentation agents open source, including the Browser agent that provides real user monitoring capabilities, allowing users to measure site performance from various browsers, devices, operating systems, and networks. By contributing to our open-source projects, users can help improve the user experience and future-proof New Relic products. The views expressed in this blog post are those of the author and do not reflect New Relic's commercial solutions or support offerings, and readers are encouraged to join the Explorers Hub for questions and support related to the content.

Bill Staples has been promoted to CEO of New Relic, taking over the role on July 1, 2021, with current CEO Lew Cirne transitioning to Executive Chairman of the Board. Cirne expresses strong confidence in Staples, citing his impressive vision for the company, particularly in driving the New Relic One strategy and transitioning to a consumption-based business model. Staples has been instrumental in advancing New Relic's product offerings, including the successful release of New Relic One and innovations like New Relic Explorer and next-gen AIOps. His leadership is expected to further the company's growth and align it with its core values. Additionally, Hope Cochran will become Vice Chair while continuing as Lead Independent Director, playing a crucial role in the company's future. This leadership transition is accompanied by optimism for New Relic's strategy and anticipated success, though it includes forward-looking statements subject to uncertainties.

Alec Isaacson discusses the challenges faced by IT operations teams in meeting business expectations due to a disconnect in how success is measured, advocating for a focus on service-level indicators (SLIs) and service-level objectives (SLOs) to bridge this gap. He explains that while IT traditionally measures resource consumption, businesses prioritize speed and quality, leading to tension and ineffective monitoring strategies. Isaacson suggests using SLIs that directly reflect service performance, such as latency and error rates, to align IT efforts with business needs and improve observability practices. He outlines the process of setting SLOs based on SLIs, emphasizing the importance of presenting them as percentages to handle statistical outliers and ensure clarity for non-technical stakeholders. This approach promises a more responsive IT service, better business alignment, and accelerated improvement, with further guidance available in the Google SRE book and through New Relic resources.

New Relic is dedicated to promoting open observability, which entails using open standards and ensuring observability functions by default for everyone. The company supports open exchange, collaborative participation, and community-driven development by open sourcing its documentation, Pixie, integrations, SDKs, and instrumentation agents, including New Relic Browser. This browser monitoring tool offers real user monitoring (RUM) capabilities for measuring site speed and performance across various browsers, devices, and networks. The commitment to open source was discussed in a Nerdlog episode by Senior Product Manager Rebecca Rodriguez and Lead Software Engineer Martin Kuba, who demonstrated how to install and configure the Browser agent, and contribute through issues or pull requests. New Relic encourages users to engage with its open source initiatives, subscribe to Nerdlog updates, and participate in the live discussions on Twitch. The blog post underscores that the views expressed are those of the author and not necessarily of New Relic, and readers are invited to join the Explorers Hub for further discussions and support.

The New Relic programmability certification course is designed to validate expertise in extending New Relic One and showcase observability skills to organizations. It assumes prior knowledge of React, Git, and GraphQL and covers topics including programmability, custom visualizations, and building on New Relic One. The course enables learners to create a New Relic One application from scratch, deploy it, and add custom visualizations to dashboards. This certification program is intended for engineers and developers who already have a foundational knowledge of New Relic fundamentals and want to validate their expertise in observability.

New Relic has introduced an observability specialist certification course aimed at validating expertise in programmability for New Relic One, targeting engineers and developers with existing foundational knowledge of the platform. This certification focuses on skills such as adding custom visualizations, creating and deploying New Relic One applications, and utilizing the New Relic One SDK's component library and CLI. The course assumes familiarity with React, Git, and GraphQL, and offers insights into creating and managing applications and visualizations within the New Relic ecosystem. The program is designed to help professionals demonstrate their in-demand observability skills, with plans for future role-based certifications such as SRE/DevOps practitioner. The initiative is part of New Relic's efforts to foster a community of engaged developers, as emphasized by Jemiah Sius, Director of Developer Relations at New Relic.

The rise of cloud computing has created a new set of skill sets required for success in modern technology environments, including technologies and platforms, integration and multi-cloud environments, trainings and certifications, organizational strategies and processes, management and negotiation, metrics and analytics, and leadership. Cloud professionals need to have a mix of technical expertise and business skills, including knowledge of cloud platforms, containerization, automation software, integration, data analysis, and financial implications. They also need to understand the importance of DevOps, Agile, Lean methodologies, and modern programming frameworks. Additionally, they must be able to communicate effectively with stakeholders, negotiate with vendors, and lead organizational strategy and decision-making processes. With the increasing demand for cloud computing expertise, aspiring professionals can capitalize on their existing skills while acquiring new ones to build a successful career in the cloud.

Cloud computing's rise is transforming required skill sets for success in technology environments, emphasizing a mix of technical and business acumen. Professionals need expertise in major public cloud platforms like AWS, GCP, and Azure, alongside skills in containerization, automation, and multi-cloud integration. Certifications from vendors and third parties enhance credentials but are not sole predictors of success. Modernizing organizational strategies, such as adopting DevOps, is crucial, as is developing management and negotiation skills for internal and external interactions. Understanding metrics and analytics is vital for cost management and application performance monitoring. Leadership in cloud adoption involves strategic decision-making, financial insight, and fostering collaboration, moving beyond traditional IT approaches to embrace innovative cloud strategies. The cloud offers career opportunities for those who adapt their skills and approach to align with its dynamic environment.

The text discusses Apache Kafka's partitioning strategy for efficient data processing and scaling. It explains that each topic is divided into partitions to distribute parallel data processing and fault tolerance. The number of partitions depends on factors like expected data volume, the number of consumers, and desired parallelism. Partitioning strategies include random partitioning, partition by aggregate (e.g., query identifier), ordering guarantee, resource bottleneck planning, storage efficiency, consumer partition assignment, sticky assignors, cooperative sticky assignors, static membership, and best practices for choosing a strategy aligned with specific use case requirements. The text emphasizes the importance of understanding data access patterns, choosing an appropriate number of partitions, using key-based partitioning when necessary, considering data skew and load balancing, planning for scalability, setting an appropriate replication factor, avoiding frequent partition changes, monitoring and tuning as needed, and adapting strategies to handle new volume and shape of data.

Apache Kafka's architecture, utilizing topics and partitions, is pivotal for efficient data processing and real-time event handling, as illustrated by New Relic's Event Pipeline team. Kafka topics, serving as data categories, can be subscribed to by consumers, with scalability achieved through partitioning, which allows parallel processing and fault tolerance. The partitioning strategy significantly influences data distribution and processing efficiency, depending on the consumer's requirements, such as aggregation needs or ordering guarantees. Different strategies like random partitioning, partitioning by aggregate, or key-based partitioning address specific needs, such as load balancing and data skew. Furthermore, Kafka offers features like StickyAssignor and CooperativeStickyAssignor to optimize partition assignment during rebalances. It’s crucial to plan partition strategies considering factors like data volume, consumer count, and resource bottlenecks while maintaining scalability. Regular monitoring and adjustments ensure optimal performance, aligning with the evolving system requirements. New Relic, an observability platform, provides tools to enhance Kafka monitoring, as demonstrated in their integration with ZenHub, underscoring the importance of a well-thought-out Kafka partitioning strategy for software success.

The New Relic Observability for Good program provides free and discounted access to its observability platform to nonprofits, including Quill.org, which aims to help all students become strong writers and critical thinkers. Quill experienced a rapid spike in users during the pandemic, with over 1 million new students signing up in under six weeks, and relied on New Relic for technical support and guidance. The organization is open source, with its core platform available under the GNU Affero Public License (AGPL) v3.0, and relies on community contributions to inform its product development. Quill's open source approach involves working closely with users and teachers to shape its content and tools, and it is now seeking volunteers with expertise in areas such as natural language processing and Ruby on Rails to contribute to its growth. The organization plans to launch new tools and APIs this year, including a focus on language analysis using AI, and aims to serve 10 million low-income students in the next five years.

The New Relic Observability for Good program supports nonprofits like Quill.org by providing access to their observability platform and AI functionality, which has been pivotal in helping Quill manage a significant surge in users during the shift to online learning. Quill.org, a nonprofit dedicated to enhancing students' writing and critical-thinking skills, experienced a rapid increase in users, reaching 3.8 million students by early 2021. Founder Peter Gault highlights the role of open source in Quill's mission, noting that their platform is open source to encourage contributions from volunteers, particularly those building their resumes. Quill has engaged over a hundred contributors and works closely with teachers to shape its products, employing tools like Slack for idea sharing and teacher advisory panels for content development. As Quill's codebase has grown more complex, attracting volunteers has become challenging, yet the organization continues to seek expertise in natural language processing and Ruby on Rails. Looking ahead, Quill plans to launch a new AI tool for analyzing students' logic and evidence in writing and is developing language-focused APIs, aiming to reach 10 million low-income students in five years.