July 2018 Summaries

In June, Gremlin, a chaos engineering startup, held a Test in Production Meetup focusing on how the company runs GameDays. Patrick Higgins, UI Engineer at Gremlin, discussed their experience with "Failure Fridays" and how valuable they have been for testing in production. The talk emphasized the importance of including people from various backgrounds, such as UI engineers, in conversations about failure to encourage cross-organization collaboration and varied perspectives. Additionally, Patrick shared resources on chaos engineering and the Chaos Community Slack channel for further learning and discussion.

In June, the Test in Production Meetup focused on chaos engineering with a talk by Nora Jones, Senior Software Engineer at Netflix. Chaos engineering is the discipline of experimenting on production to find vulnerabilities in systems before they become unusable for customers. At Netflix, this is done through a tool called ChAP (Chaos Automation Platform), which allows users to inject failures into services and test assumptions about those services. The goal of chaos engineering at Netflix is to improve availability by proactively finding vulnerabilities in services using live production traffic. Safety and monitoring are crucial aspects of testing in production, as they help contain the blast radius and ensure a good observability story. ChAP has found numerous vulnerabilities within the Netflix ecosystem, leading to improvements in service resilience and customer experience.

In a recent Test in Production Meetup, Ted Strzalkowski from LinkedIn discussed how his team focuses on resilience engineering and chaos engineering. The Waterbear Team at LinkedIn aims to build a culture of resilience by creating an automated, comprehensive, and measurable resiliency feedback loop through education, validation, and frameworks. They use tools like LinkedOut, their failure injection framework, which supports multiple failure modes, ad hoc testing, and automation for continuous testing. The team is also working on implementing screenshots, custom failure detection, reporting, and absolute latency support in the future.

LaunchDarkly introduces new billing and usage metrics to help customers better understand their application's usage of the feature management platform. The new metrics can be found under Account Settings in the dashboard, providing information on monthly active users (MAUs), projects, environments, seats, server connections, and more. Enterprise plan users also have access to Experimentation and Data Export metrics. These updates aim to improve user experience and understanding of how applications are using LaunchDarkly.

LaunchDarkly is making infrastructure changes starting July 16th to improve scalability and resilience of its public-facing edge infrastructure, which will result in changing public IP addresses. Most customers won't need updates, but some with firewalls or external proxies may require IP address whitelisting. LaunchDarkly has created a new endpoint that reports all public IP address ranges for their services, allowing automatic updates to IP-based whitelists as infrastructure changes occur. This information is documented in the API reference guide and status page updates will be posted as new IP address ranges are added. Customers with questions can contact [email protected].

LaunchDarkly has introduced a new 'filters' feature to help users manage their numerous feature flags. Users can now filter flags by status, type, and tags created by their team. This tool helps in identifying and eliminating flag debt in the code. Additionally, flags can be sorted by oldest, newest, or alphabetical order for easy access. The filters feature is now available on LaunchDarkly's dashboard to improve efficiency in managing feature flags.