September 2018 Summaries

The author recently joined LaunchDarkly, where they experienced an efficient onboarding process and were quickly integrated into the development cycle. One notable difference from other dev teams is the ability to use their product as a direct consumer, allowing them to ship code with feature flags for potential rollback if needed. Additionally, engineers at LaunchDarkly practice end-to-end ownership of code, which incentivizes high-quality work and service robustness. The company culture seems enthusiastic and close-knit, with team members willing to take on a wide range of tasks related to software development.

LaunchDarkly has introduced new flag-level insight graphs that provide detailed information on how users interact with feature flags in an application. The Flag Insights graph displays the number of times a flag is fired, the variations evaluated for each flag, and any changes made to the flag. This helps engineering and product teams understand user behavior and make informed decisions about their LaunchDarkly implementation. The graphs also include annotations based on audit log entries, allowing users to connect changes with user experiences. These insights can help identify issues, improve performance, and enhance user experience. The new feature is available for all customers under the Insights tab for each flag.

LaunchDarkly introduces two new controls to enhance communication within teams when making flag changes. The first control allows administrators to make comments required for flag or segment changes in specific environments, while the second control adds an extra confirmation step for critical environment flag changes. These features aim to reduce the likelihood of incorrect changes and increase team confidence as LaunchDarkly scales across projects and teams.

LaunchDarkly is a reliable service that prioritizes availability and correctness. Over 40 billion feature flags are evaluated daily on the platform, ensuring customers can depend on it. The company has worked hard to create a robust service against various types of failures over the last four years. LaunchDarkly SDKs streamline integration by fetching all feature flags and evaluation rules automatically when an application starts. Flags and evaluation rules are cached locally in applications, reducing performance overhead. In case of failure or worst-case scenarios, the platform ensures that applications continue to perform as expected using locally cached values. LaunchDarkly also offers LD-Relay for additional redundancy, which can be deployed in a customer's environment and uses an optional Redis cache for storing flag evaluation rules. The streaming connections provided by LaunchDarkly ensure fast and reliable delivery of updates to flag or evaluation rules across all connected SDKs. This is crucial for microservice architectures and distributed systems that require consistent application of rules.

Many large enterprises have successfully implemented DevOps practices to improve software development and delivery processes, despite challenges such as legacy assets and rigid organizational structures. One example is Watchful Bank, which underwent a digital transformation that included reorganizing its teams into cross-functional units, implementing containerization for rapid deployment, automating testing, and using feature flags to manage risks associated with legacy systems. As a result, the bank achieved faster release cycles and improved quality while minimizing disruptions to customer experience and back-office functions.

In episode 48 of To Be Continuous, Edith Harbaugh interviews Paul Biggar about his fourth startup, Dark. The company aims to remove all complexity from building software by tackling infrastructure, API, deployment, and code complexities. They started with an MVP that took two weeks to build and have since grown the team to four people. One of their key learnings is that small UI improvements can have a significant impact on user experience when the product is at the core of someone's workflow.

In this talk, Daniel Mor from Waze discusses monitoring infrastructure for large-scale data centers. He emphasizes that traditional per-instance monitoring becomes problematic as systems grow larger and more complex. Instead, he recommends a "large-scale way" of monitoring, which involves placing a mid-layer between the monitoring service and clients. This mid-layer, or monitoring gateway, holds all logic and can support various features such as aggregating metrics and publishing API's for data fetching. By using this approach, businesses can maintain control over their monitoring systems while ensuring that they are scalable and adaptable to future needs.

At a recent Meetup, Matt Duftler from Google and Michael Graff from Netflix discussed testing in production with systems and processes at scale. They focused on automated canary deployments, which involve exposing a small number of users to a change before rolling it out to a broader audience. The speakers explained the concept of canary deployment and how Spinnaker and Kayenta make this possible. Automated Canary Deployment consists of provisioning a new baseline group running exactly what's already in production, alongside the canary group, and using configuration to determine significant metrics for comparison.

At the July Test in Production Meetup, Girish Patangay from Facebook discussed how the company manages different versions of its platform using Gatekeeper. He explained that each user has a slightly different version of the software due to this tool, which is used throughout their entire stack. The talk covered various uses of Gatekeeper, including testing and rollout, policy and laws compliance, load testing, and experimentation. Patangay also introduced Quick Experiment, a tool built on top of Gatekeeper that enables developers to launch experiments quickly and efficiently across all platforms. He highlighted the importance of limiting audience size for experiments, dividing user bases into segments, and using real-time analytics tools like Scuba to analyze results. The challenges discussed included managing costs associated with experiments and dealing with lingering issues on mobile devices.

In this episode of the Product Love podcast, Adam Gross discusses his experience as a product manager and how he believes that product management is an empathy provider role. He talks about the importance of trust in organizations and how it can be built through transparency, visibility, and accountability. Additionally, he shares some of his favorite books for people who want to become great product managers, including Innovator's Dilemma by Clayton M. Christensen, Inspired by Marty Cagan, The Inmates Are Running the Asylum by Alan Cooper, The Four Steps to The Epiphany by Steve Blank, and Technologies of Freedom by Ithiel de Sola Pool.