July 2022 Summaries

In the July edition of their newsletter, incident.io introduces a new feature - an on-call calculator that simplifies the process of determining on-call pay by integrating with PagerDuty accounts and accounting for different payment scales. They also highlight "The Practical Guide to Incident Management," a comprehensive guide covering all aspects of responding to incidents, and mention Chris Evans' appearance on the Last Week in AWS podcast discussing incident management topics. Additionally, they announce their recent $34 million Series A raise aimed at expanding international operations and bringing increased resilience to organizations worldwide. They also invite customer referrals with a reward of a $200 Amazon voucher.

Post-mortems can be valuable tools for learning from incidents and improving future performance. However, they are often executed poorly, leading to wasted time, unhelpful discussions, and a lack of actionable change. To avoid these pitfalls, consider automating the process of gathering information, focusing on higher levels of reflection, steering discussions towards forward-looking solutions, and seamlessly integrating action points into issue trackers for prioritization and follow-up.

Incident.io has raised $34 million in funding to help organizations improve their resilience against failures and complications in technology. The company assists businesses in treating every failure as an opportunity for learning and growth, turning resilience into a superpower. With this investment, they plan to expand internationally, accelerate product plans, and grow their team. They have already gained popularity among hundreds of fast-growing tech companies across various industries. Their product allows organizations to spot issues before they happen, fix them faster when they do, and improve communication and coordination. The company is also opening an office in New York City as part of the expansion plans.

The author reflects on a past decision to build an expenses software instead of purchasing one, highlighting several mistakes made during this process. They underestimated the complexity of the task and found it increasingly difficult to maintain as time went on. Additionally, they failed to consider the opportunity cost of building an internal system rather than focusing on their company's core product. The author also mentions that leaving a legacy behind after departing from the company made maintenance even more challenging. They conclude by discussing when it is appropriate to build something in-house and emphasize the importance of considering the value of one's time before making such decisions.

Incident.io introduces its first bug tracker integration with Sentry. Users can now link Sentry issues to incidents and attach them when a Sentry link is posted in an incident Slack channel. Additionally, activity on the Sentry Issue will be propagated to the incident Slack channel for real-time updates.

This post discusses how to implement tracing for Gorm database queries using OpenCensus and Google Cloud Tracing libraries. The author explains how to use Gorm's Callbacks API to hook into the query lifecycle, allowing them to pull out data for observability purposes. They demonstrate how to start and end spans in their tracing system, as well as how to make these spans more informative by adding table names, query fingerprints, line of code that called the query, WHERE params from select queries, and number of rows affected. The author also provides a link to the full code implementation for this process.

The author shares their experience leading a major incident at work for the first time. They emphasize the importance of clear communication and collaboration within the team, especially when dealing with technical issues that may be difficult to understand. Key takeaways include documenting all discussions in the incident channel, maintaining calmness as an Incident Lead, providing regular updates, taking breaks when necessary, and using this experience as a valuable learning opportunity for both experienced and inexperienced responders.