April 2019 Summaries

The New Relic C SDK has been released to provide production visibility for applications written in C and C++ languages, which are used extensively in software development due to their focus on performance and efficiency. The SDK is designed to instrument C applications, allowing developers to gain insight into performance issues that can result in significant cost savings and deliver a competitive advantage. The release of the C SDK as an open-source project makes it easier for customers to adopt, test, distribute, and deploy it across various platforms. With the C SDK, developers can control exactly what gets instrumented, add custom attributes, and report transaction tracing and other real-time metrics vital to understanding application health and performance.

New Relic has launched a C SDK for its APM 360, expanding application performance monitoring to C and C++ applications, which are fundamental in many core and complex systems, including IoT devices. This SDK joins New Relic's existing agents for languages such as Java, PHP, and Python, and aims to enhance visibility and performance insights in traditionally opaque areas of software development. The C SDK is open-source, encouraging community contributions and allowing users to instrument applications directly, offering transaction tracing, error troubleshooting, and custom metrics. This hands-on approach gives users fine-grained control over what aspects of their applications are monitored, facilitating the linkage of performance data to business metrics. The SDK's release highlights the enduring importance of C and C++ in modern software, alongside emerging languages like Rust, and supports New Relic's mission to provide comprehensive monitoring across diverse programming environments.

The world's most iconic brands are great companies because they aspire to do great things and have inspiring missions that drive them to be better and put their customers front and center. World-class companies become great when their employees find meaning in the work they do and have a sense of ownership, driven by setting ambitious goals and creating a shared sense of purpose. New Relic's North Star is shining light on the internet, aiming to instrument the digital world and establish standards for measuring and improving software performance. Despite the industry's rapid growth, only 5% of applications are currently instrumented, leaving companies in the dark about their software's performance. Tackling this big job, New Relic is pursuing a more perfect internet to help customers create better software, experiences, and businesses.

Iconic companies like Amazon, Nike, and BMW thrive by having ambitious missions and inspiring their employees with a shared sense of purpose, which serves as their guiding principle. Similarly, New Relic aspires to perfect the digital landscape by standardizing how software performance is measured and improved, addressing the significant gap where only 5% of applications are currently instrumented. As the global economy becomes increasingly digitized, New Relic aims to provide comprehensive visibility into software performance to drive business growth, emphasizing that software should be seen as a key driver of success rather than merely a cost. Under the leadership of Lew Cirne, New Relic is committed to illuminating the complexities of the internet to enhance user experiences and business operations, highlighting the importance of measurement in achieving improvement.

New Relic’s container orchestration team has developed best practices for maintaining integrity in services running on their platform. They emphasize the importance of health checks, which should respond with a status code representing the true state of the service's health. Health checks should be designed to ensure a service is "running" and not just responding. Latency, context, dependency handling, cold starts, and thundering herds are key considerations when writing effective health checks. Additionally, readiness checks provide a signal that a deployment is safe to continue, while monitoring and alerting solutions help keep the platform observable and operators happy. The CF team advises developers to use active external checks, such as New Relic Synthetics, to verify the network path for HTTP services. The CF metrics pipeline collects container events, metrics, and logs, powering key alerts that all services should use in production. By following these guidelines, teams can build highly available services that delight customers and keep operators rested and healthy.

New Relic's Container Fabric (CF) team focuses on deploying and maintaining containerized services through a platform that manages around 1,000 machines, utilizing physical hardware across multiple data centers. The team emphasizes the importance of health and readiness checks, monitoring, and alerting to ensure the reliability and observability of services, which contributes to customer satisfaction and operator well-being. They have developed best practices for maintaining service integrity, such as defining health checks to represent a service's true state, considering latency and dependency handling, and managing cold starts and thundering herds. Additionally, the CF platform uses a metrics pipeline to gather container data, enabling the creation of key alerts for CPU usage, out-of-memory events, SIGKILLs, and excessive load-balancer connections to prevent service disruptions. By fostering a culture of resilience and observability, the CF team aims to keep operators rested and customers happy, believing that robust architecture practices directly impact service level agreements and customer satisfaction.

The key to success in today's complex technology environments is not just "failing fast," but rather "learning how to fail better." This approach involves implementing a strategic process where iteration and experimentation contribute directly to meaningful business outcomes and excellent customer experiences. A successful example of this approach is the New Relic engineering team, which built and launched a major new feature in just three months by implementing five key best practices: making technology choices that promote a fast-paced development process, giving teams and product managers a shared understanding of priorities and success metrics, choosing a team structure that promotes trust and teamwork, employing a launch process that supports iteration and incremental delivery, and relying on critical visibility and instrumentation tools like New Relic. By adopting these best practices, organizations can use failure as a catalyst for achieving success consistently, sustainably, and in ways that matter to their business.

In the dynamic field of software development, the concept of "fail fast" is often promoted as a way to quickly iterate and learn from mistakes, but this approach must evolve into "failing better" to drive meaningful business outcomes. This involves making strategic technology choices, fostering a collaborative team environment, and employing processes that support iterative development. A case study of the New Relic engineering team illustrates this approach; they successfully launched a major feature, the metric explorer, in just three months by implementing best practices such as containerization, microservices architecture, shared success metrics, incremental delivery, and real-time feedback mechanisms. The success of this project was attributed to a combination of smart technology use, a cohesive team structure, and strategic experimentation, demonstrating that failure can be a powerful catalyst for innovation and success when approached with the right mindset and tools.

The key points of this text revolve around common myths and misconceptions surrounding incident response in IT operations, DevOps, and SRE teams. These myths often lead to ineffective incident response processes that prioritize speed over quality, overlook the importance of proactive learning and prevention, and fail to recognize the value of minor incidents for improving overall system robustness. The article highlights eight common myths, including the idea that "speed is everything," that only customer-impacting incidents matter, and that a blameless culture means no accountability. It emphasizes the importance of focusing on effectiveness, transparency, and proactive learning in incident response processes to improve overall system reliability and reduce downtime.

The New Relic Query Language (NRQL) allows users to query and analyze data from their applications, providing insights into performance, usage, and other metrics. NRQL enables users to simplify data, ignore irrelevant information, transform data, facet data, combine unexpected combinations, put barriers around data, and apply advanced mathematics. With these features, users can gain a deeper understanding of their application's behavior, identify trends and patterns, and make data-driven decisions. The article provides seven tips and tricks for advanced NRQL querying, including using functions like `disguise`, `ignore`, `transform`, `facet`, `combine`, `barrier`, and `mathematics` to extract valuable insights from the data.

Incident response teams in DevOps and site reliability engineering face challenges in managing incidents efficiently due to common myths and misconceptions that hinder their effectiveness. Key myths include the belief that speed is the sole priority, neglecting the importance of thorough and sustainable solutions, and that addressing incidents ends once the immediate issue is resolved, overlooking the value of postmortems and learning from incidents to prevent future occurrences. Furthermore, the misconception that only major, customer-impacting incidents require attention can limit learning opportunities from smaller, internal incidents. Effective incident response requires transparent communication, not just focusing on the mean time to resolution (MTTR) but also on improving processes through metrics that reflect the quality of problem statements and proactive learning. The notion of a blameless culture is emphasized, which fosters accountability without fear of blame, encouraging transparency and improvement across the organization. Additionally, while detection is improving with automated tools, the challenge remains in efficiently mobilizing the right people to resolve incidents, highlighting the potential of AI and machine learning to streamline response efforts. The article advocates for a distributed incident response capability across IT teams, ensuring engineers are well-trained and empowered to make critical decisions during incidents.

Kevin Scaldeferri, a Principal Software Engineer at New Relic, explores the advanced capabilities of the New Relic Query Language (NRQL) for querying and analyzing data collected from applications. By providing various tips and tricks, he demonstrates how users can disguise, ignore, transform, and facet data, as well as use unexpected combinations and advanced mathematical functions in their queries. Scaldeferri emphasizes the versatility of NRQL in reshaping data displays, grouping data by different axes, and applying statistical functions, all while assuming a foundational understanding of NRQL's syntax and components. He encourages users to experiment with these techniques to gain deeper insights into their data and to share their findings in the New Relic Explorers Hub, although he notes that his views are personal and not necessarily those of New Relic.

Kubernetes is a container orchestration technology that has gained widespread adoption in organizations adopting container-based application architectures. The challenges of monitoring Kubernetes performance and health are significant as environments scale and become more complex, making it harder to answer basic questions about cluster health and entity relationships. New Relic's Kubernetes cluster explorer is an innovative tool addressing these challenges by providing a curated-visualization platform that filters, sorts, and searches for Kubernetes entities, understands relationships and dependencies within an environment, and employs data-visualization techniques to give customers fast and intuitive answers about their Kubernetes environments. The cluster explorer provides improved views into Kubernetes health and performance, faster and more effective troubleshooting, and is designed to be accessible to a wide range of team members in various roles.

Kubernetes, a rapidly growing container orchestration technology, presents challenges in monitoring performance and maintaining visibility as environments scale and become complex. New Relic addresses these challenges with its Kubernetes cluster explorer, a tool that enhances the monitoring capabilities of the New Relic platform by providing advanced visualization and exploration of Kubernetes environments. This tool allows users to efficiently troubleshoot and optimize performance by visualizing cluster health through concentric rings that represent different levels of detail, from nodes to pods and containers. The cluster explorer offers multi-dimensional views, enabling teams to quickly identify dependencies, performance issues, and potential problems before they impact customers. By simplifying the complexity of Kubernetes environments, the cluster explorer supports proactive monitoring and effective troubleshooting, catering to the needs of both developers and operations teams.

This is a software risk analysis process that involves identifying, assessing, and prioritizing potential risks associated with a software project. The goal is to anticipate and mitigate potential problems that may arise during the development, deployment, or maintenance of software. Key steps in this process include identification of risks, risk assessment, risk mitigation planning, monitoring and control, and reviewing the risk matrix regularly to adapt to evolving risks and ensure that mitigation strategies remain effective. A risk matrix is a tool used to categorize risks along two axes: Impact and Likelihood; and teams should prioritize risks based on severity and likelihood of occurrence. The process aims to improve the chances of successful project outcomes by enhancing the ability to identify, assess, and manage risks, and providing best practices for software development teams to follow. Regular review and updates of the risk matrix are crucial to ensure that mitigation strategies remain effective and that the team can adapt to evolving risks. By following this process and using a risk matrix, teams can proactively anticipate and address issues before they escalate, making informed decisions, allocating resources effectively, and establishing mitigation strategies to handle unforeseen circumstances. This elevates the visibility of risks, allowing the organization to make better decisions as it prioritizes work. The ability to prioritize reliability work that is in service to reducing documented risk recognizes that reliability delivers business value.

Software risk analysis is a critical component of risk management in software development, aiming to identify, assess, and mitigate potential challenges that could impact project success. The process involves identifying risks related to technical, operational, organizational, and external factors, assessing their probability and impact, and developing mitigation strategies. New Relic highlights the importance of using a risk matrix, a tool that categorizes risks by likelihood and impact, to improve team alignment and prioritize reliability work. By using methodologies like Threat Assessment and Remediation Analysis (TARA), teams at New Relic categorize risks into "Avoid," "Reduce," and "Accept," enabling them to address risks effectively and maintain system reliability. Regularly reviewing and updating the risk matrix ensures that mitigation strategies remain relevant, while stakeholder involvement, clear documentation, and contingency planning enhance the overall risk management process. Ultimately, prioritizing reliability work not only reduces risks but also adds business value, as exemplified by New Relic's approach to integrating risk analysis into their operational practices.

To monitor and understand the performance of a Kubernetes environment, one must rethink their monitoring strategies to account for new layers introduced in complex, distributed environments. To get started, deploy the New Relic Infrastructure agent for your Kubernetes cluster, which provides visibility into operational data like resource usage, namespaces per cluster, and container restarts. The Kubernetes integration dashboard offers a good starting point for monitoring the cluster, but the cluster explorer provides deeper insights into performance and dependencies across the entire environment. With the cluster explorer, users can troubleshoot with advanced filtering options, view detailed information about pods and containers, and monitor performance metrics from applications hosted in Kubernetes.

New Relic's Complexity in Context video series, particularly the second episode, focuses on monitoring Kubernetes environments using New Relic to manage the intricacies of distributed technologies in modern software setups. The episode guides users through deploying and configuring the New Relic Infrastructure agent for Kubernetes, which provides access to crucial metrics such as container CPU and memory usage, and container restarts. After deployment, users can explore operational data through the Kubernetes integration dashboard, offering insights into resources, namespaces, and container statuses. For more detailed analysis, the Kubernetes cluster explorer enables users to observe performance and dependencies, providing advanced filtering options to troubleshoot issues effectively. This integration also facilitates monitoring of applications hosted within Kubernetes by linking infrastructure metrics with application performance metrics, thereby ensuring a comprehensive view of the environment. New Relic's Kubernetes monitoring is available to Infrastructure Pro level customers, with additional resources accessible for further learning and support.

Faster transaction times can lead to improved business outcomes, but it's essential to prioritize customer experience and focus on availability and correctness before optimizing for speed. Ensuring that applications and infrastructure are consistently and reliably available is crucial, as a single outage or error can have significant impacts. Correctness also matters, as even fast systems can be rendered useless if they're not functioning correctly. To optimize for performance, it's necessary to identify critical customer journeys, understand response-time sensitivity, and consider perceived load times. Measurement and instrumentation are key to identifying KPIs, setting baselines, and assessing the business impact of optimization efforts. By focusing on these best practices, organizations can go faster without spinning their wheels and drive meaningful business outcomes.

Improving the speed of online customer transactions can lead to significant business outcomes, as demonstrated by examples from companies like Staples, Walmart, and Trainline, where enhancements in page-load times resulted in increased conversions and revenue. However, the pursuit of speed should not overshadow the importance of ensuring system availability and correctness, as even fast systems can fail if they are not reliable or produce errors. Businesses are encouraged to focus on optimizing customer experiences by identifying key customer journeys, assessing response-time sensitivity, and considering perceived load times. Measurement and observability are crucial in this process, with tools like New Relic providing comprehensive insights into both frontend and backend performance, helping companies to prioritize and address performance challenges effectively. While optimizing transaction speeds offers immediate benefits, embracing a customer-centric approach can uncover even greater opportunities for improvement.