August 2025 Summaries

In December 2024, Scribe was launched to assist responders by automatically transcribing incident calls, reducing the need for manual note-taking and preserving important context for analysis. Despite its popularity, the feature faced challenges, such as requests for bug fixes, additional functionality, and broader platform support. An impact review was conducted to evaluate Scribe's performance and identify areas for improvement, revealing high adoption rates on supported platforms, though issues with permissions and configuration hindered broader use. The analysis showed Scribe was particularly valuable for high-severity incidents but identified that it only successfully joined about half the calls due to potential trust issues and technical barriers. Recommendations included enhancing user control, improving setup guidance, and expanding platform compatibility. Some updates, such as manual controls for call participation and error messaging improvements, were already implemented, while future plans include supporting more platforms and languages. The review underscored the importance of regular evaluations to align product development with user needs.

Incident management tools in 2025 are expected to feature AI-powered investigation, integrated on-call scheduling with intelligent routing, chat-native collaboration in platforms like Slack or Teams, built-in status pages for stakeholder communications, and automated post-incident insights. These tools aim to reduce Mean Time to Resolve (MTTR), enhance stakeholder communication, and facilitate continuous learning from incidents. The integration of these features in a unified platform eliminates context switching and reduces operational overhead, with AI providing context-aware assistance to accelerate incident response. Companies like incident.io exemplify this all-in-one approach, offering comprehensive solutions that consolidate multiple functionalities into a single platform, thereby improving efficiency and reducing costs. As AI becomes a standard component in enterprise service management, platforms that incorporate it effectively will be crucial in maintaining system reliability and customer trust.

Oli Monaghan-Coombs, a product engineer at incident.io, shares insights on how using Claude, an AI tool, has significantly enhanced his onboarding process and productivity. Having joined the company recently, Oli describes how Claude has helped him understand the codebase through CLAUDE.md files, which provide a structured explanation of the code architecture. He highlights how Claude assists in visualizing event flows, debugging issues with the help of Linear's MCP, and building a comprehensive knowledge base through note-taking and diagramming. Oli also discusses the benefits of customizing Claude’s settings to improve its functionality, emphasizing how this AI tool allows him to address questions independently, thus enabling him and his colleagues to focus on delivering value to customers. Despite the challenges of adapting to new coffee machines, Oli expresses appreciation for the supportive onboarding experience facilitated by Claude.

AI-powered Site Reliability Engineering (SRE) tools are transforming how DevOps teams manage incidents by shifting from reactive to proactive, data-driven decision-making. These tools, including platforms like incident.io, PagerDuty AIOps, Datadog Bits, Resolve.ai, and BigPanda Autopilot, leverage artificial intelligence to predict failures, correlate events, and execute remediation actions autonomously, reducing cognitive overload and operational toil. The transition to AI-driven SRE is driven by the increasing complexity of modern infrastructures, such as microservices and multi-cloud environments, which traditional tools cannot efficiently manage. The adoption of AI SRE tools offers significant improvements in operational metrics, including alert noise reduction, faster incident resolution, and decreased manual intervention, ultimately leading to enhanced reliability, cost savings, and engineer satisfaction. Successful implementation requires careful tool selection based on organizational needs, phased deployment to build trust, and awareness of potential pitfalls like over-reliance on black-box models and hidden costs. As AI SRE tools become integral to incident management, they offer a competitive advantage through improved system resilience and efficiency.

Incident.io has developed an efficient API setup using Goa, a design-first API framework, to address the challenges of scaling their product's API infrastructure. Initially, their API setup involved manually written handlers and ad hoc struct definitions, which became unsustainable as the company grew. By leveraging Goa, they transitioned to a structured approach where APIs are defined in a Domain Specific Language (DSL), allowing Goa to generate boilerplate code, HTTP handlers, validation logic, typed service interfaces, OpenAPI documentation, and basic Go clients automatically. This transition has streamlined their API development process, reduced the risk of errors, and allowed for more focus on business logic rather than boilerplate code. Additionally, incident.io enhances Goa's output by generating specialized client libraries for different audiences, including TypeScript interfaces for the frontend and lean mobile clients, ensuring type safety and efficient API interactions. This automated and modular approach has enabled them to move faster, focus on product development, and maintain a scalable API infrastructure.

Tom Wentworth details his experience with "vibe coding," a rapid prototyping approach he used to create an Apple TV app utilizing the incident.io API, which he describes as an intuitive and well-designed tool for quick development. Despite lacking professional coding skills, Wentworth, a marketer, successfully built an "incident command center" dashboard for conference rooms, emphasizing that vibe coding prioritizes swift functionality over polished or production-ready code. The process involved creating a tvOS app that displays real-time incident data in a visually engaging way, using a video game design mindset to ensure usability with limited controls. By allowing for dual theme support and iterative, creative coding, vibe coding encourages exploration and learning about APIs' capabilities in a hands-on manner. Wentworth encourages others to experiment with the incident.io API, highlighting its flexibility and ease of use for building unconventional yet functional applications.

For engineers aspiring to advance beyond the Senior level to roles like Staff or Principal Engineer, the journey involves cultivating a blend of experience, expertise, and wisdom, collectively known as the trifecta. This development is not simply about completing tasks or projects but requires navigating complex challenges and making strategic decisions that benefit the company in the long term. Senior+ engineers act as leaders by projecting technical vision, multiplying the capabilities of others, and setting examples through clear communication, resilience, and adaptability under pressure. They prioritize the company’s success over individual achievements and demonstrate positivity and grit, acting as role models who inspire higher standards within their teams. The ability to operate without detailed guidance, charting one's path in ambiguity, is crucial, as it reflects a high level of proficiency and the capacity to drive organizational success.

Anthropic's Model Context Protocol (MCP) is a new standard designed to connect AI assistants like Claude to external data sources, allowing these AI systems to perform tasks such as incident management seamlessly through a structured and secure interface. The incident.io MCP server, built using Golang, serves as a bridge, enabling Claude to access and interact with incident.io accounts to manage, create, update, and analyze incidents through natural language commands. This open-source project is described as "vibe-coded," indicating it was developed out of personal interest rather than customer demand, and offers a conversational interface that translates user requests into API calls, providing an intuitive method for incident handling. The server is available on GitHub, allowing users to configure and use it with Claude Desktop, while the developers encourage contributions and feedback from the community to enhance its functionality.