October 2021 Summaries

Effective communication with customers during an incident is crucial for maintaining trust and strengthening relationships. Five steps to better customer communication include providing context to minimize speculation, explaining what actions are being taken, setting expectations for resolution time, directing customers on appropriate actions, and informing them about the next update. Companies such as Heroku, GitHub, Cloudflare, Monzo, and GitLab exemplify good incident communication practices.

In this text, the author discusses some pitfalls that can occur during postmortem analysis of incidents. They emphasize the importance of assuming good intent when investigating an incident and understanding the motivations behind actions taken by individuals involved. The author also highlights the inevitability of future incidents and suggests that learning from them is more valuable than trying to prevent every possible issue. Additionally, they recommend taking a soak time before committing to action items and focusing on the process of running debriefs rather than just the written artifact.

Tom Blomfield, founder of Monzo and GoCardless, shares his experience dealing with a major incident that led to a 10-hour outage at Monzo. He emphasizes the importance of having clear roles and responsibilities during an incident, as well as effective communication protocols. The company's biggest learning was the need for good incident response tooling, which eventually led to the development of incident.io, a Slack bot that streamlines incident management processes.

Stephen and Gerhard Lazu discussed incident workflows, simplicity, and how incident.io uses incident.io during an episode of Ship It! Podcast. They emphasized the importance of encoding processes in advance to focus on problem-solving when incidents occur. The ideal incident flow depends on each company's needs but should include clear roles, structured actions, and good communication. Simplicity is a key aspect of incident.io, with a single Slash command for users to get started. They use the product liberally for various interruptions and have a simple production setup. The platform aims to provide rails for organizations to follow their processes and will soon introduce workflows for defining incident management procedures.