September 2022 Summaries

Incident.io has had a busy month with new hires, blog posts, and feature updates. They made their on-call calculator free for everyone to use, allowing users to create monthly pay reports easily. Additionally, they introduced customizable incident lifecycles, enabling organizations to set timestamps for each status and store structured data for reporting and analysis. Support for Splunk On-Call escalations was also added, allowing users to call in a lifeline during incidents directly from Slack. A blog post about building workflows at incident.io was published, providing insights into the engineering team's problem-solving approach. Lastly, Stephen, co-founder and CEO of incident.io, shared his start-up journey on the Best Work Podcast.

Incident.io, an incident management platform, has recently celebrated several milestones including its first year anniversary for two employees, moving into a fourth London office in the past 12 months, raising $34 million in Series A funding, and opening its first international office in the US. The company has seen significant growth in usage of its product features such as workflows, which automate tasks like sending Slack messages or emails during incidents. Since December 2021, approximately 65% of customers have created workflows that have automated over 27,000 tasks and actions. Additionally, the platform has helped manage around 18,000 declared incidents across hundreds of organizations.

The author started their new job at incident.io, beginning with a breakfast meeting with other new starters. They received laptops and swag upon arrival at the office. Onboarding sessions were conducted in person, unlike their previous remote role. A team lunch was organized by the office manager. The week included 1:1 meetings with colleagues to understand their roles better. A fire alarm incident occurred during the first week, followed by a queue for lifts that led to climbing stairs to reach the office. On Thursday evening, they played rounders and later visited a pub. The week ended with a team update session where each department discussed their achievements and plans for the next week. Overall, the author is excited about starting their second week at incident.io.

Aaron has recently joined incident.io as a Product Engineer after working at Fintech companies like Bloomberg, Oaknorth, and Freetrade. He co-founded Courtsite.my, a leisure sports booking platform in Malaysia. Excited to wear both product and engineering hats, he aims to improve incident management's user experience. In his free time, Aaron enjoys photography, tennis, sim-racing, movies, and exploring food options.

Anna is a Product Manager who recently joined incident.io after working at Snyk, an open source security startup. Previously, she was a freelance developer with experience in various industries such as healthcare, education, open source, and government. In her free time, Anna enjoys reading books, learning new languages, and doing jigsaw puzzles.

Incident.io held its first annual intracompany rounders match at Victoria Park with twenty employees participating. Team 1 emerged victorious by scoring 10 rounders to Team 2's 9.5. The event featured a mix of company leaders and new hires, showcasing the teamwork and spirit within the organization. Despite some initial setbacks, both teams displayed impressive skills on the field. The match concluded with a friendly rematch where Team 2 emerged victorious, though the validity of that innings is contested.

Cybersecurity threats are prevalent in modern businesses, with 93% of company networks being vulnerable to breaches. Establishing an incident response team is crucial for minimizing damage from cyberattacks and preventing future incidents. Key roles on such a team include an on-call team, incident leads, communications leads, incident response timeline leads, security analysts, and legal representation. To optimize the team's performance, it is essential to define roles and responsibilities, provide training, set up communication channels, establish relationships with external parties, and use automated tools.

At incident.io, the Workflows product helps automate organizations' processes in responding to incidents. The workflow engine consists of core concepts such as triggers, conditions, and steps. Triggers represent lifecycle events that can trigger workflows, while conditions apply to details associated with the trigger. Steps are actions taken by a workflow, which may be parameterized by values from the scope. Resources represent variable types available in the workflow program and implement a common interface. Workflow steps also implement an interface and register themselves with a global registry of steps. The Workflow Builder uses information exposed by the API to power the UI, allowing customers to configure triggers, conditions, and steps for their workflows.

This article discusses the development of a workflow engine and its executor in incident.io's platform. The author provides an overview of how they built the workflow engine, including the data model, control over workflow execution frequency, handling concurrent execution, and tracking progress. They also evaluate the project against their goals, discussing build effort, onboarding and defects, extensibility, and robustness. The workflow engine has become a core part of incident.io's platform, enabling various product features and improving developer velocity.

In this text, the author shares their experience as employee #2 at a scaling startup called incident.io during its first year of operation. They discuss three key learnings from their journey: prioritization is crucial for managing customer feedback and shipping features; leverage is critical in laying foundations that accelerate development; and everything changes every three months, requiring constant adaptation to the evolving product, customer base, and team dynamics. The author also emphasizes the importance of a motivated, talent-driven team focused on shipping high-quality products.

Incident.io is a platform designed to help organizations manage incidents by engaging their entire organization, integrating with multiple tools, and providing features such as timeline creation, post-mortem document management, and trend analysis. However, it may not be the ideal fit for companies looking for an SRE-only tool, a Slack bot, or those with fewer than 20 employees. Additionally, if your organization values customization over predefined processes or prioritizes minimizing MTTR above other aspects of incident management, you might want to consider alternative solutions.

Incident.io has made their on-call pay calculator free for everyone to use, regardless of whether they are using the platform or not. The calculator provides full support for timezones and helps visualize pay calculations, catching errors around special rates or holidays. It also sends each user a breakdown of their shifts, allowing them to understand and predict their pay. Users have praised the tool for its accuracy and ease of use, stating that it has made their lives easier.

The importance of incident timelines cannot be overstated, as they provide valuable insights into what happened during an incident and allow teams to learn from their mistakes. To effectively use an incident timeline, it's essential to ensure its accuracy and usefulness by constructing a story around the key events that occurred, getting everyone on the same page, identifying key discussion points, and ensuring that all stakeholders are kept up-to-date with relevant information. A useful timeline should tell a story backed by an audit trail, capture all events, including gaps in activity, and be a living document that can be extended during a debrief to find new insights. By creating a thorough and accurate incident timeline, teams can maximize their learnings from incidents and improve their response to future events.