February 2020 Summaries

With a significant increase in serverless usage volume, traditional architecture and infrastructure are becoming less relevant for many developers. The migration to serverless is delivering benefits such as cost reduction, scalability, decreasing barriers to entry, and increasing developer diversity. According to Farrah Campbell, Ecosystems Director at Stackery, serverless should be the primary solution for most projects, with alternatives used only where necessary due to its high ROI. As organizations adopt serverless, they face challenges in replicating success on a wider scale, developing workflows and tools, and overcoming stereotypes around its limitations. However, with larger adoption, serverless is expected to overcome these challenges, grow in wider adoption, and increase diversity among developers. The author believes that serverless has lowered barriers to entry, increased diversity, and provides a high ROI, making it the preferred solution for most projects.

The rapid growth in serverless technology, marked by a 209% increase in usage in 2019, is reshaping the future of software development by offering benefits such as reduced costs, scalability, and increased developer diversity. Farrah Campbell, Ecosystems Director at Stackery, advocates for serverless as a primary solution for most projects, emphasizing its role in simplifying infrastructure management and lowering entry barriers for developers. Despite past technical challenges, serverless has evolved to handle a wide range of use cases, prompting larger organizational adoption and dispelling myths about its limitations, such as the misconception that it only relates to functions. As organizations increasingly question their investments in traditional server operations, serverless adoption is expected to grow, particularly in environments traditionally resistant due to cold start issues. The ongoing advancements in serverless platforms continue to excite industry leaders, as they facilitate the building of scalable and resilient applications while fostering a more diverse developer community.

The author of this text shares a personal story about their struggles with mental health and how they found solace in therapy. They highlight the importance of addressing mental health stigma, particularly in the black community, where there is often reluctance to discuss these issues due to fear of being perceived as weak or broken. The author notes that corporate America is slowly starting to address this issue, but more work needs to be done to reduce stigma and provide support for employees struggling with mental health challenges. The text also touches on the importance of building a culture of authenticity and inclusivity in the workplace, where employees feel supported and empowered to bring their authentic selves to work.

New Relic's focus on authenticity and support for mental health was highlighted during Black History Month with an event featuring Jewel Love, a psychotherapist and founder of Champions of Mental Health, who addressed the stigma surrounding mental health in the Black community. This event, co-sponsored by the Relics of Color and NeuRelics Employee Resource Groups, aimed to provide tools for inner peace to Black executives and their families, and to foster discussion on mental health challenges in corporate America. The event underscored the importance of reducing the stigma around mental health, particularly in the Black community, where individuals may fear being perceived as weak. Participants shared personal experiences, noting the difference between therapy and talking to friends, and highlighted the therapeutic benefits of feeling understood and less isolated. New Relic supports mental health through insurance benefits, a helpline, and an employee assistance program, while also advocating for wellness strategies like yoga and meditation. The event was part of a broader Black History Month initiative at New Relic, including enriching programs and community-building activities, reflecting a commitment to supporting Black mental health and fostering a connected workplace culture.

The integration of Vector with Timber.io and New Relic allows for a streamlined observability practice by collecting and analyzing data from various sources in a single platform. Vector, an open-source tool, enables the creation of custom observability pipelines with sources, events, transforms, and sinks. The New Relic Logs integration simplifies this process, allowing users to collect metrics and logs from their systems and analyze them on the New Relic One platform. By reducing complexity and providing a unified view of system performance, this integration can help organizations improve their overall architecture and observability.

Ashley Jeffs from Timber.io and Julian Giuca from New Relic discuss the collaboration between New Relic and Timber.io, which resulted in integrating New Relic Logs with Vector, an open-source tool for constructing observability pipelines. This integration simplifies data management by allowing the collection and transformation of logs and metrics from various sources such as StatsD, Syslog, Docker, and Kafka, into the New Relic One platform for analysis. Vector enhances logs and metrics with metadata through its transform feature, and its pipelines consist of sources, events, transforms, and sinks, which facilitate data processing and storage. To utilize this integration, users can install Vector on multiple operating systems and configure it to send data to New Relic Logs by specifying a data source and a New Relic Logs sink in the configuration file, requiring a New Relic license key. The blog emphasizes that the opinions expressed are those of the authors and not necessarily those of New Relic, and it encourages joining the Explorers Hub for further discussion and support.

To understand application performance monitoring (APM), one must grasp its significance in today's digital age, where speed is critical for business success and customer satisfaction. APM delivers real-time data about web application performance and user experience, helping identify potential problems before they affect end users. It involves five vital components: user experience monitoring, code-level diagnostics, key business transactions, application infrastructure deep dive, and supporting infrastructure health. By automating root cause diagnosis and streamlining troubleshooting, APM empowers teams to take meaningful strides toward optimal performance, providing a single source of truth with easy access to critical information and actionable insights.

The blog post introduces a new series titled "The Fundamentals," aimed at engaging non-technical business users in technical topics, specifically focusing on Application Performance Monitoring (APM). APM, pioneered by New Relic in 2008, is crucial for maintaining a fast and error-free digital experience, as businesses increasingly rely on digital platforms for success. The post outlines the importance of APM in identifying performance bottlenecks and errors in applications, highlighting its role in connecting user experience with application performance to resolve issues quickly. APM provides real-time data on application performance, helping developers diagnose problems and improve user satisfaction, and is often compared to observability, which offers deeper insights into why issues occur. The author emphasizes the value of APM in enhancing application performance through user experience monitoring, code-level diagnostics, transaction tracking, and infrastructure analysis, ultimately contributing to improved business outcomes and customer experiences.

The text discusses events as a critical telemetry data type for observability, providing discrete and detailed records of significant points in real-time. Events can be used to track specific transactions, errors, and deployments, helping with fine-grained analysis and troubleshooting issues. However, storing large volumes of event data can consume significant storage resources. The text introduces the events-to-metrics service, which converts event data into aggregated dimensional metrics for long-term storage and analysis, providing more efficient data storage and faster results on data queries and chart creations. This service allows users to identify trends and anomalies by analyzing KPIs, query converted metrics, and build dashboard widgets alongside existing events, logs, traces, and third-party metric data. The service works by creating and managing event-to-metrics rules using the New Relic GraphQL API, enabling users to create new metrics from events, delete existing events-to-metrics rules, enable or disable events-to-metrics rules, and query metrics using the New Relic Query Language (NRQL) or chart builder.

Events serve as critical telemetry data for observability by providing detailed records of significant occurrences, aiding in real-time analysis and troubleshooting. New Relic's events-to-metrics service offers a more efficient storage and analysis method by converting event data into aggregated metrics, which reduces storage costs and enhances long-term trend analysis. This service allows for the creation and management of conversion rules using the New Relic GraphQL API and enables querying with the New Relic Query Language (NRQL). The conversion process retains the original event data while generating metrics for 13 months, facilitating faster data queries and dashboard widget integration. This approach is particularly beneficial for tracking high-cardinality data, such as transaction response times, thus providing a more streamlined way to analyze key performance indicators over extended periods.

The serverless landscape is evolving, with certain developmental gaps closing, and competition driving innovation in vendors and tools. However, organizations face challenges in adopting serverless, including training existing staff, embracing infrastructure-as-code, and getting senior management on board. The key to success lies in starting small, iterating, and having data to support the benefits of serverless. Despite some myths surrounding serverless, such as "lock-in" and cold start issues, the technology is maturing, with enhancements from cloud vendors addressing these concerns. As a result, serverless adoption is expected to continue on a steady trajectory upward in 2020.

In 2020, the adoption of serverless technology is gaining momentum despite challenges such as training existing staff in new methodologies and integrating infrastructure-as-code (IaC) into traditional development processes. Dave Townsend, a Principal Software Engineer at Matson, emphasizes the importance of starting small and iterating as organizations transition to serverless. While there are technical hurdles, like operational governance and the need for more account access, cloud vendors like AWS are closing gaps, particularly with improvements in Lambda startup time in VPCs. The serverless landscape is steadily evolving, with advancements in frameworks like Serverless Framework, SAM, AppSync, and Amplify, which simplify the development for frontend developers. Townsend also highlights the growing excitement around event-based architectures enabled by services like EventBridge, and dispels myths about serverless, such as the misconception of “no servers,” noting that the reduction in cold-start issues is helping to alleviate concerns.

Hybrid cloud computing combines public and private clouds to provide a flexible and secure IT infrastructure for organizations. A hybrid cloud solution joins two individual cloud systems together, allowing companies to store sensitive data on a private cloud while leveraging functionalities and resources from a public cloud service. This approach offers several benefits, including retaining control over select data, taking advantage of the expansive power of public cloud services, and meeting specific geographic compliance requirements. Hybrid clouds are used in various industries, such as financial services, e-commerce, healthcare, and more, to address unique challenges and requirements. To optimize performance, security, and efficiency, it's essential to implement hybrid cloud best practices, including defining clear objectives, consistent management, security and compliance, data integration and interoperability, scalability and elasticity, hybrid cloud networking, backup and disaster recovery, cost management, training and skill development, performance monitoring and optimization, vendor management, and continuous evaluation and adjustment. However, a hybrid cloud deployment model may not be suitable for all businesses, as it can affect requirements for monitoring application and infrastructure performance.

The blog post introduces a series aimed at non-technical readers to explain technical topics, with a focus on hybrid cloud computing as a prevalent solution for businesses worldwide. Hybrid cloud solutions integrate public cloud services, such as AWS, Azure, or GCP, with a private cloud to manage data across diverse environments through encrypted connections, allowing companies to retain control over sensitive data while leveraging public cloud resources. This model, distinct from multi-cloud, offers benefits like scalability, compliance with geographic regulations, and reduced latency, making it ideal for industries like finance, e-commerce, and healthcare. Best practices for managing a hybrid cloud include setting clear objectives, ensuring security and compliance, integrating data seamlessly, and utilizing scalable architecture, although the suitability of hybrid cloud solutions varies by business. The post is authored by Lee Atchison, a cloud computing expert, and emphasizes that while hybrid cloud offers significant advantages, it isn't a one-size-fits-all solution, and decisions should be tailored to specific business needs.

There is a growing trend of serverless adoption among enterprises, with a 206% increase in average weekly invocations over the last 12 months and a 178% increase in functions per account. Developers are primarily using Node.js and Python for building serverless applications on AWS Lambda, but Java adoption is expected to increase due to improvements such as Provisioned Concurrency and VPC enhancements. The report highlights key trends, including smaller function code sizes, continued use of older languages like Node.js 6.10 and Python 2.7, and the need for tools to support widespread serverless adoption. Serverless experts discuss challenges such as complexity, organizational adoption, and gaps in services, but also note progress made by AWS and the growing community. The report aims to provide knowledge-based decisions about architecture and performance goals, ultimately building better software.

The "For the Love of Serverless: 2020 AWS Lambda Benchmark Report" provides insights into the growing adoption of serverless computing, particularly within enterprises, highlighting a 206% increase in average weekly invocations and a 178% rise in functions per account over the past year. Despite the benefits of serverless, such as increased agility and scalability, challenges remain, including the complexity of developing applications that interact with multiple managed services and organizational hurdles in adopting serverless as a primary architecture. Key industry voices like Jeremy Daly and Farrah Campbell discuss the evolving landscape and the need for tools to support central operations teams. The report focuses on AWS Lambda, noting the popularity of Node.js and Python for serverless applications and anticipating increased Java adoption due to AWS's improvements in cold start times and VPC functionalities. The report is based on trillions of serverless events processed by New Relic One, with findings and trends to be discussed in a webinar featuring industry experts.

Serverless adoption is rapidly increasing, with nearly 50% of companies planning to switch to serverless architecture in the next year, according to Forrester. This trend is driven by its benefits of increased agility, scalability, and efficiency for customer-facing applications and critical workloads. The phenomenon of serverless has prompted a report highlighting key trends, insights, and benchmarks essential for anyone building a serverless architecture. Experts, including Sheen Brisals from The LEGO Group, share their perspectives on the current state of the industry and its direction, discussing challenges such as organizational mind-shifts, vendor lock-ins, tooling inconsistencies, and cost complexities. Despite these challenges, experts are excited about the future of serverless, with advancements in event-driven architecture, consolidation of platforms, and a growing community. The report aims to debunk common serverless myths, including the notion that it brings complete cost savings, eliminates vendor lock-in, or is a magic solution for monolith problems.

Serverless architecture has been gaining traction among development teams due to its agility, scalability, and efficiency, with a significant increase in adoption reported by companies like New Relic and Forrester. The "For the Love of Serverless: 2020 AWS Lambda Benchmark Report" aims to provide insights and benchmarks on this technology, highlighting conversations with industry experts such as Sheen Brisals from The LEGO Group. Brisals discusses the organizational challenges of shifting to serverless, such as the difficulty in direct cost comparisons with traditional systems and the complexities surrounding support solutions like monitoring. He also addresses technical challenges like tooling inconsistencies and the confusion stemming from multiple service offerings. The future of serverless appears event-driven, with Amazon EventBridge playing a central role, and the community is moving toward understanding the deeper reasons for adopting serverless beyond its basic definition. Despite the perceived cost benefits and concerns over vendor lock-in, the technology offers broader organizational advantages, and myths like "serverless makes you debt-free" and "Lambda functions solve all problems" are being debunked by experts in the field.

New Relic simplifies log management by providing fast, reliable, and highly scalable solutions for collecting, managing, and analyzing log data from various sources. It integrates with popular open-source logging tools like Fluentd, Fluent Bit, Logstash, and Amazon CloudWatch, allowing developers and operations teams to gain deeper visibility into application and infrastructure performance data. By bringing contextual data to the logging experience, New Relic enables users to correlate log data with other telemetry to reveal meaningful patterns and trends in their applications and infrastructure. The company's logs in context feature adds metadata that links log data with related data like error or trace information, allowing users to quickly identify and resolve issues by narrowing down from all logs to the exact log lines needed. New Relic offers a Kubernetes integration that provides in-depth information about cluster performance and reports on data and metadata about nodes, namespaces, deployments, ReplicaSets, pods, clusters, and containers. The company also provides plugins for FluentBit and Logstash to capture Kubernetes logs and send them to New Relic. By enabling logs in context for Java applications running in Kubernetes clusters, users can correlate log messages with related error traces or distributed traces, making it easier to troubleshoot production incidents and connect their log data with other telemetry data.

The blog post discusses the challenges of managing and analyzing the increasing volume of log data in the context of software development and IT operations, and how New Relic's log management solutions address these issues by offering fast, reliable, and scalable log management integrated with application and infrastructure performance data. It highlights New Relic's ability to reduce mean time to resolution (MTTR) by connecting log data with other telemetry data, such as metrics and traces, and introduces the concept of "logs in context," which adds metadata to link log data with related information to quickly diagnose and resolve problems. The article provides a detailed guide on setting up logs in context for a Java application running in a Kubernetes cluster, utilizing integrations with common logging tools and New Relic's Kubernetes integration for enhanced visibility and troubleshooting capabilities. It emphasizes the benefits of using New Relic to quickly identify and resolve issues in Kubernetes environments by correlating log data with application activity, and concludes with insights into monitoring logs in Kubernetes and troubleshooting applications efficiently, highlighting the expertise of the author, Alec Isaacson, in application and infrastructure monitoring.

WP Engine is a managed WordPress host that powers 35% of all websites on the internet and drives more of the most heavily trafficked sites than all other managed WordPress hosts combined, with over 100,000 customers operating 500,000 websites globally. To manage this growth, WP Engine uses observability tools like New Relic to ensure peak performance at all times, as even a nanosecond slowdown can affect conversions. By integrating New Relic's monitoring solution into their DXP, WP Engine has developed a cohesive data-driven culture that enables them to deliver insights and build experiences easily, quickly, and securely for their customers. This partnership has allowed WP Engine to remain nimble and flexible while experiencing hypergrowth, and has also enabled them to document performance improvements and verify significant gains for some customers of up to 40%. Additionally, New Relic provides a single source of truth for the different layers of WP Engine's cloud platform, allowing global teams to see the same information at the same time. The partnership has also led to the creation of a joint application performance monitoring solution with New Relic specifically for WordPress products, which is now being used by thousands of users through WP Engine.

WP Engine, a leading provider of managed WordPress hosting solutions, leverages New Relic's observability tools to ensure optimal performance and scalability for its vast clientele, which includes over 100,000 customers operating 500,000 websites globally. As WordPress's influence expands, powering 35% of the internet, WP Engine uses New Relic to gain critical insights into application performance, facilitating a data-driven culture that enables rapid response to customer needs and supports hypergrowth. The integration with New Relic provides a unified view of operations across both AWS and Google Cloud environments, significantly enhancing infrastructure performance and reliability. This collaboration not only helps WP Engine maintain cutting-edge performance but also allows it to extend these benefits to its partners through a joint application performance monitoring solution specifically designed for WordPress. As a result, WP Engine has been able to deliver substantial performance improvements, offer comprehensive training, and ensure customer satisfaction with real-time data insights, thus solidifying New Relic's role as a pivotal partner in its strategic growth and operational success.

The New Relic telemetry data platform is designed to provide a unified view of an organization's technology stack, enabling the identification, understanding, and resolution of issues that impact business operations. The platform adheres to three design principles: observability requires a unified telemetry platform, real-time investigation requires both speed and flexibility, and dynamic demand requires unlimited scalability. The platform achieves millisecond performance through optimized query routing, parallelization, and in-memory caching. It also provides capabilities such as a single query interface, intelligence for correlating insights across data sources, and elasticity to scale business operations. With its multi-tenant architecture, the platform delivers predictable costs and supports customers with growing demand. The views expressed on this blog are those of the author and do not necessarily reflect the views of New Relic, and any solutions offered are environment-specific and not part of the commercial solutions or support offered by New Relic.

New Relic's telemetry data platform is designed to alleviate launch-day anxiety by providing a unified platform for observability, enabling real-time investigation, and offering unlimited scalability. This platform combines metrics, events, logs, and traces to deliver a complete view of a technology stack, allowing users to quickly identify and resolve potential issues without the need to navigate multiple systems. By utilizing a schema-less database, New Relic offers both speed and flexibility, allowing users to conduct ad hoc queries quickly. The platform's architecture supports millisecond performance by optimizing for speed and parallelization, ensuring efficient data processing and query execution. Additionally, New Relic's scalability accommodates unpredictable demand spikes, with the ability to ingest over twelve billion data points per minute, offering a single query interface and intelligence features that correlate insights across data sources. The platform provides elasticity and predictable costs, allowing businesses to scale confidently while only paying for necessary resources.

Vic Soares, a Director of Product Management at New Relic, discusses how W3C Trace Context is making distributed tracing easier to implement, more reliable, and ultimately more valuable for developers working with modern, highly distributed applications. This standard simplifies use cases where developers instrument services using tools from different distributed tracing solutions, enabling cross-vendor interoperation of traces. With the launch of support for W3C Trace Context in New Relic's APM agents, developers can now easily integrate their existing tracing tools and enjoy a more seamless observability experience within the New Relic platform. The standard defines two standardized context HTTP headers that serve to propagate context correlation information between services: `traceparent` and `tracestate`. This common format enables trace propagation across other trace instrumentation that conforms to the standard, clearing barriers for middleware vendors to support propagating trace headers, and for framework vendors to build in tracing instrumentation. By adopting W3C Trace Context, developers can overcome the challenges of mutually incompatible header formats and broken traces, leading to more flexibility and fewer barriers to observability. New Relic's open instrumentation initiative aligns with this standard, providing APIs, Telemetry SDKs, and exporters to meet customer needs for interoperation between vendors and open source tools. The company is committed to the W3C group and will continue to provide composable instrumentation solutions that seamlessly work with open standards.

Distributed tracing is crucial for developers managing microservices applications, but inconsistent header formats have complicated its implementation across different tools. The W3C Trace Context standard addresses this issue by establishing a uniform method for passing contextual information via HTTP headers, thereby enabling interoperability among various tracing tools and enhancing observability. New Relic, participating in the W3C Trace Context Working Group, has integrated support for this standard into several of its APM agents to facilitate seamless trace propagation across services. This standardization not only simplifies cross-vendor tracing by defining a common format for headers but also supports the New Relic platform's open instrumentation initiative, allowing users to incorporate tracing telemetry from diverse sources. The introduction of W3C Trace Context is expected to increase flexibility and reduce barriers to effective observability, aligning with New Relic's commitment to open standards and expanding distributed tracing capabilities.