January 2019 Summaries

AWS re:Invent is a prominent cloud computing event that attracts tens of thousands of attendees each year. Despite its popularity, many enthusiasts missed out on the opportunity to attend the event and experience New Relic's Lightning Talks, which covered various topics such as Kubernetes, microservices, cloud migration, and more. To address this, New Relic has compiled a YouTube playlist featuring 13 recorded Lightning Talks, including talks from customers like Morningstar and Xero, and experts in the field. These talks showcase best practices for building and maintaining modern software systems, overcoming common challenges, and leveraging tools like New Relic to drive technical change and business value.

AWS re:Invent is a significant event in the cloud computing calendar, attracting numerous cloud enthusiasts to Las Vegas every year. New Relic, a prominent participant, offers a series of lightning talks at the event, covering varied topics like Kubernetes, microservices, and cloud migration strategies. For those who cannot attend, New Relic provides recorded sessions on a YouTube playlist, featuring insights from experts and partners on subjects such as transaction trace support, cloud platform monitoring, data-driven changes in SaaS organizations, and DevOps integration. Highlighted talks include discussions on microservices architecture, cloud migration success, and the benefits of integrating New Relic with various platforms like Atlassian and PagerDuty, offering valuable guidance for navigating the complexities of modern cloud technologies.

New Relic Synthetics offers powerful application monitoring and troubleshooting capabilities that enable teams to simulate real user flows and behavior at any frequency from locations around the world. The guide provides insider tips and best practices for organizations just starting their journey with Synthetics, including setting up a naming convention for synthetic monitors, setting appropriate permissions, and using the REST API to manage monitors programmatically. Secure credentials are also essential for storing sensitive data within Scripted Browsers and other scripted monitoring tools deployed across the Synthetics environment. Understanding New Relic Synthetics data is crucial, including interpreting failures, scripts, and logs to diagnose and respond to monitor failures. Insights queries can be used to make data actionable, while custom alerts using NRQL can set up alternate alert configurations. The guide also provides resources for learning more about Synthetics best practices and tips.

New Relic Synthetics offers comprehensive application monitoring and troubleshooting capabilities by allowing users to simulate real user behavior across various digital touchpoints, helping to identify and solve issues before they affect actual users. To optimize the use of Synthetics, the New Relic Community Team provides guidance on best practices, such as establishing naming conventions for monitors, setting appropriate permissions, utilizing a REST API for monitor management, and securing sensitive data with encrypted credentials. Understanding Synthetics data involves interpreting error codes, troubleshooting script failures, and making data actionable through insights queries and custom NRQL alerts. The blog also encourages users to explore additional resources and community-generated content on Synthetics best practices through the New Relic Explorers Hub.

New Relic University has launched a new Learning Management System (LMS) to provide self-serve training for its customers, offering a greater variety of courses and improved search capabilities. The new system allows users to learn on their own schedule and pace, with the ability to start a course and pick up where they left off later. Interactive learning formats and progress tracking are also available, making it easier for users to engage with the content. The expanded course catalog includes featured courses such as Certified Performance Pro and Getting Started with APM, which provide in-depth training on New Relic's platform and tools. Users can access the new LMS at learn.newrelic.com and create their own account to start learning.

New Relic University (NRU) has launched an enhanced Learning Management System (LMS) to improve self-serve training for its customers, offering a wider variety of courses, better search capabilities, and interactive learning formats that allow users to learn at their own pace. The new platform has led to an over 80% increase in new accounts and course enrollments, highlighting its appeal. Users can track their progress, access an expanded course catalog, and engage in practical, hands-on learning using their own data. Despite needing a separate login for the LMS, New Relic plans to introduce single sign-on in the future. Featured courses include certifications in New Relic APM and infrastructure, as well as insights into using various New Relic APIs. The platform also offers live and recorded webcasts for flexible learning options. For inquiries, the NRU team is available via email, and users are encouraged to join discussions at the Explorers Hub for additional support.

ScribbleLive added New Relic to see and fix issues that took days to resolve, allowing them to reduce churn and transform engineering from a cost center into a thought leader within the organization. With customized dashboards using performance data, ScribbleLive can now give customer success managers more visibility into what's happening with their customers, support teams more tools at their disposal, and leadership more actionable insights to inform business decisions. The company has seen measurable outcomes in faster troubleshooting and lower churn rates, and is now exploring the role of AI and other advanced technologies to drive innovation transformation. By demystifying data and contextualizing it around clients, ScribbleLive can enable the broader organization to function differently and think differently about how they do their day-to-day operations.

ScribbleLive, a marketing-technology and content-creation company, implemented New Relic to enhance its operational efficiency and foster meaningful business conversations across different levels of the organization, from customer success managers to executives. The integration of New Relic was pivotal in reducing issue resolution time from days to minutes, transforming the engineering team into a strategic asset rather than a cost center, and reducing customer churn. New Relic's tools, such as real-time dashboards and APM 360, provided ScribbleLive with actionable insights, facilitating improved customer engagement and support. The initiative, supported from the leadership level, emphasized the importance of data-driven decision-making and the necessity of breaking down silos within the company. This transformation allowed ScribbleLive to leverage engineering data to drive change, enhance customer relationships, and maintain a competitive edge by turning IT into a source of strategic advantage.

The text discusses the concept of "adversarial game days" in software development and reliability engineering, which involves intentionally introducing faults into a system to test its resilience and recovery capabilities. New Relic, a software company, uses this approach as part of their site reliability best practices. The goal is to identify potential weaknesses in the team's collective knowledge and improve the mental model of the system. Adversarial game days can be run in various environments, including non-production versions of the system, and tools such as Toxiproxy can be used to inject faults into the system. The exercise should be conducted like an actual incident, with a scribe taking notes and customer-support representatives participating, and a retrospective held after the exercise to discuss what could have been done better and what was learned.

New Relic employs "adversarial game days," a variant of chaos engineering, to assess the resilience and reliability of their systems by simulating failures and observing team responses. These exercises, inspired by the concept of deliberately injecting faults into systems, help reveal gaps in the team's mental models and operational processes, allowing them to better prepare for real-world incidents. Game days facilitate cross-functional collaboration, testing communication tools and incident response strategies in environments that mimic production settings. The outcomes of these simulations highlight areas for improvement, such as enhancing monitoring and alerting systems, refining incident response protocols, and updating documentation, ultimately aiming to reduce mean time to resolution (MTTR) during actual system failures. These exercises not only strengthen the team's understanding of the systems but also encourage the sharing of insights and best practices across teams, fostering a culture of continuous improvement and preparedness.

---` The New Relic Container Fabric team developed an inventory management solution using Kubernetes API server and custom resource definitions to manage their containerized services in a dynamic and rapidly scaling environment. They were struggling with the current approach, which required splitting code paths based on infrastructure providers and had operational bottlenecks due to increased secure network connections across multiple regions. The team looked for solutions but found that existing options were flawed. Instead, they leveraged Kubernetes' integral configuration service, creating custom resources and a fetcher service to enumerate hosts from each infrastructure provider and update the kube-apiserver with the required data. This solution provided a single source of truth for inventory management, improved production-quality performance, and enabled locally meaningful and extensible object representation. The implementation was simple, and the team found that adapting Kubernetes to serve as an inventory-management database had a significant impact on their operational capabilities, enabling them to define declarative desired states for their cluster and build cluster services in a decomposed way.

Inventory management is a crucial yet challenging task for New Relic's Container Fabric team, which manages a rapidly growing infrastructure of about 1,000 machines across multiple regions. Initially relying on Mesosphere DC/OS for container orchestration, the team faced increasing complexity in inventory management, prompting them to explore various solutions. They ultimately adopted the Kubernetes API server for a more efficient and scalable inventory management system, leveraging its custom resource definitions to maintain an up-to-date picture of their infrastructure. This new approach reduces operational bottlenecks by eliminating the need for VPN connections and enabling dynamic configuration changes, thereby enhancing their capability to operate at scale. The solution also lays the groundwork for future applications, such as automated monitoring and change orchestration, as the team plans to transition fully from DC/OS to Kubernetes.

We worked to automate the build-and-deploy process for our Windows agent using a combination of tools including Jenkins, Ansible, Powershell, and WinRM. Our goal was to cover four essential steps: building the agent installer, deploying it to canary machines, running tests on those machines, and releasing the installer. We used Ansible to automate these tasks by leveraging its existing support for Windows and our team's existing use of it for provisioning machines. Powershell was chosen as a complementary tool for remote access and deployment. After overcoming the initial challenge of introducing new technologies to the team, we successfully reduced the release time of our Windows agent from a half-day manual process to a five-minute Jenkins job.

New Relic's team, primarily operating in Linux, faced challenges in automating the build-and-deploy process for their Windows Infrastructure agent, which was originally manual and labor-intensive. By leveraging tools like Ansible, PowerShell, and Windows Remote Management (WinRM), they successfully automated the pipeline, significantly reducing the release time from a half-day manual process to a five-minute Jenkins operation. This new process involved building an agent installer, deploying it to canary machines, running tests, and releasing it, with Ansible playing a key role due to its existing use in provisioning and compatibility with Windows. Despite initial challenges with PowerShell and WinRM, these tools proved to be user-friendly, and their implementation led to a streamlined and efficient deployment process. The author, Ruben Hervas, emphasizes that while these solutions were effective for their specific environment, they may not reflect the official solutions offered by New Relic, and discussions for further support are encouraged on the Explorers Hub.

New Relic Mobile provides real-time monitoring and management capabilities for iOS and Android applications, allowing developers to uncover performance issues, understand user interactions, and find and fix problems quickly. To get the most out of New Relic Mobile, it's essential to create a clear naming schema for mobile apps, keep agents up to date, understand data interpretation, and use features like crash report symbolicating, HTTP requests analysis, and custom data capture to troubleshoot issues. By following best practices and tips outlined in this guide, developers can turn insights into action and optimize their New Relic Mobile setup.

New Relic Mobile is a tool designed to help organizations monitor and manage their iOS and Android applications by providing detailed real-time performance data, tracing errors, and analyzing app throughput. It allows developers to gain code-level visibility into user interactions, helping them understand app performance from the user's perspective. The platform emphasizes best practices such as creating a consistent naming schema for apps, keeping mobile agents updated to leverage new features and bug fixes, and using symbolicating tools to translate crash reports into human-readable formats. New Relic Mobile also aids in diagnosing third-party service slowdowns by analyzing HTTP requests and enables custom data capture to enhance app insights. By utilizing features like the crash event trail and breadcrumb events, developers can effectively troubleshoot app crashes and optimize performance.

Chaos engineering is a discipline that involves proactively causing failures in random places at random intervals throughout systems to build confidence in their ability to withstand turbulent conditions in production. This approach was first adopted by Netflix, which migrated from an on-premise stack to a distributed cloud-based architecture and introduced tools like Chaos Monkey to test the resilience of its systems. Chaos engineering is not about randomly causing chaos but rather about carefully injecting harm into systems to test their response to failure. The goal is to minimize the blast area and ensure that business operations are not significantly impacted. It involves planning meticulously, having the right team in place, and simulating scenarios that have the potential to make systems become unavailable or cause performance degradation. Chaos engineering can be thought of as a formal method to generate new knowledge about complex systems and uncover systemic weaknesses. By using chaos engineering, companies can improve the reliability of their modern architectures and prepare for outages before they occur.

There is a growing discussion about the role of early career engineers in modern software development organizations, with companies like New Relic recognizing their value and creating programs to support them, such as the Ignite program, which aims to increase opportunities for early career engineers by reducing cost, risk, and delivering high-performing engineers from the first day. The program is designed to provide full-time engineers from day one, with extra support, mentoring, and coaching, as well as exposure to a broad technical overview of the company's systems, and will be organized into cohorts with two phases, allowing engineers to experience different technologies and team dynamics. The goal is to deliver a smooth transition to a new role at the end of the program, and New Relic plans to expand the program in 2020 to bring in more engineers from within and outside the company.

Netflix, initially known for streaming services, pioneered the field of chaos engineering to enhance the resilience of its complex technology infrastructure. This approach emerged after Netflix transitioned from on-premise servers to a cloud-based architecture on Amazon Web Services (AWS) following a major outage in 2008. The company developed Chaos Monkey, a tool that intentionally introduces failures to test system robustness, leading to the birth of chaos engineering as a discipline focused on experimenting with distributed systems to ensure they can withstand disruptions. Contrary to its name, chaos engineering involves meticulously planned experiments rather than random disruptions, aiming to uncover system vulnerabilities and improve reliability. This practice is now adopted by major companies like Google and Amazon. Experts in the field emphasize the importance of understanding system complexities and conducting controlled experiments to gain insights and prepare for potential outages, turning chaos engineering into a method for learning and enhancing system resilience rather than merely testing for failures.

New Relic has launched the Ignite program, an incubator initiative designed to support early career engineers by offering full-time positions as Associate Software Engineers (ASE) and providing them with mentoring, coaching, and a broad exposure to the company's systems. The program aims to enhance diversity and create a clearer pathway for individuals entering the tech industry, addressing barriers faced by new engineers, especially those from non-traditional backgrounds. ASEs participate in two phases: initially rotating through different teams to gain a comprehensive understanding of the technical ecosystem, and subsequently focusing on their area of interest with guidance from a technical specialist mentor. The pilot cohort begins in April with internal candidates, while a second cohort will start in July, involving external hires. The program is part of New Relic's broader strategy to improve the diversity and quality of its engineering talent, ultimately transitioning ASEs to Software Engineer 1 roles within the company.

FlixBus and Onefootball presented their digital transformation journeys around DevOps and customer experience at FutureStack18 Berlin, discussing the importance of monitoring and tracing for autonomy, scaling customer experience through application monitoring, and re-architecting for innovation with cloud migration and Agile methodologies. The event included presentations by FlixBus' Eric Heymann on the company's technical journey towards autonomous engineering teams, Onefootball's Fred Wynyk on their success at scaling during the 2018 FIFA World Cup Russia, and Thomas Blood from AWS on leveraging the cloud to change IT's role in the business. The insights shared by these companies offer valuable lessons for innovating faster and scaling customer experience.

FutureStack18 Berlin was an event held on December 5, 2018, at Academie Lounge, where leading German companies such as FlixBus, Günstiger, Onefootball, Seven Senders, and OLX shared their experiences with digital transformation, particularly focusing on DevOps and customer experience. The gathering featured customer presentations, Q&A sessions, and a panel on DevOps best practices, illustrating how these approaches have facilitated innovation and scalability. Key presentations included FlixBus's emphasis on monitoring and tracing for autonomous engineering teams, Onefootball's use of Kubernetes and New Relic for infrastructure visibility during the 2018 FIFA World Cup, and AWS's insights on cloud migration and the transformation of IT from a cost-center to a business enabler. The event provided a day of learning, networking, and sharing of valuable insights on optimizing digital strategies and leveraging technology to enhance business operations.

The New Relic platform is built around six core principles that define the role of monitoring in a modern enterprise: prioritizing application performance over technology stack, providing true Software-as-a-Service (SaaS) architecture to empower teams without burdensome tasks, focusing on modern software development methods and technologies, offering intelligent analytics to find and fix issues fast, being open and extensible for integration with existing workflows, and providing easy adoption and implementation with world-class support.

In the quest to modernize software stacks, New Relic emphasizes the importance of focusing on customer experience rather than solely on new technologies like cloud services and containers. The company outlines six core principles for effective monitoring in contemporary enterprises, centered around customer-centric application performance, leveraging true SaaS architectures, and embracing modern software practices like DevOps and serverless computing. New Relic provides a comprehensive APM solution that offers deep instrumentation and analytics, enabling quick identification and resolution of issues, supported by an open and extensible platform that integrates with existing workflows. With a commitment to ease of adoption, New Relic ensures its monitoring platform is user-friendly, scalable, and backed by robust support, aiming to help teams deliver a consistently excellent customer experience.

Starting a programming job can be daunting, even for those with a computer science degree or boot camp certification. The author highlights five technical skills that every beginning developer should possess: version control systems, HTML and CSS basics, developer toolkits (IDEs, editors, CLI tools), regular expressions, and SQL. Additionally, the author emphasizes the importance of soft skills such as clear and effective written communication, using libraries and existing code, defensive programming, debugging, and teamwork. The author also stresses the need for continuous learning and improvement, including seeking help when needed, understanding how to work on existing code, and contributing to open-source projects. Overall, the article aims to provide a comprehensive guide for new developers to get started with software development and improve their skills over time.

Published initially in 2014 and updated in 2019, this post addresses the gap between formal education and real-world software development challenges, highlighting essential skills often overlooked in academic and boot camp curriculums. It categorizes these skills into technical, mindset, habits, and soft skills, emphasizing the importance of understanding version control systems, HTML and CSS basics, and using developer toolkits. It also stresses the need for mastering SQL and regular expressions as well as valuing defensive programming, debugging, and teamwork. The post underscores the significance of effective written communication and leveraging existing code, advocating for a mindset of continuous learning and collaboration. It proposes practical educational improvements like assignments that mimic real-world tasks and encourages contributing to open-source projects to gain experience. The author, Andy Lester, offers additional insights through recommended readings and emphasizes the ongoing nature of learning in the software development field.