September 2025 Summaries

Being on-call can be challenging for engineers, often resulting in disrupted sleep and alert fatigue due to frequent, noisy alerts. To address this issue, an AI-powered On-call Assistant Bot was developed to handle alerts by performing an initial triage and responding only to significant signals. Integrated into Slack, the bot assesses alerts, providing context and recommended actions for genuine incidents, while muting unnecessary noise to prevent future disturbances. The workflow involves several components, including AI agents and automation tools like n8n, to seamlessly connect and manage alerts. The project showcases the potential of combining AI with observability workflows to reduce engineer burnout and improve efficiency. Through real-world examples, the bot demonstrates its ability to distinguish between signal and noise, gaining trust by explaining its decisions. The initiative also highlights a growing trend among organizations to develop AI solutions that alleviate operational burdens, with future plans to enhance the bot's capabilities further.

Observe has developed an AI Site Reliability Engineer (AI SRE) to help streamline the troubleshooting and incident response process, allowing engineers to focus more on development rather than firefighting issues. This AI tool analyzes observability data, enabling developers to resolve issues independently, which reduces the number of engineers needed for incident response, thereby shortening mean time to recovery (MTTR) and decreasing on-call burdens. The AI SRE is designed with features like a chat interface for asking questions in natural language, automatic dataset selection, and built-in root cause analysis and postmortem drafting. It can be integrated into the existing Observe UI or embedded into IDEs and chat applications. In a practical demonstration, AI SRE successfully identified the root cause of a payment failure in a demo app and calculated the resulting revenue loss, demonstrating its capability to provide quick, accurate insights. Additionally, it can generate monitoring alerts to proactively address recurring issues. This tool is available to existing Observe users participating in the MCP Server private preview, with plans for broader access in the future.