Building On-call: Our observability strategy

At incident.io, they prioritize excellent observability to ensure high availability for their customers. They achieve this by creating a user-focused lens on their internal setup, treating it like a product with great UX. Their system follows the principle of "alerts in, notifications out" and is structured around four key areas: alerts, alert routing, escalations, and notifications. Each area has an overview dashboard that provides a high-level view of the system's health, followed by more specific dashboards for each subsystem. They also use event logs to provide a single, consistently formatted log line for each task, and tracing to visualize what a request spent its time doing. The goal is to make their observability setup feel great, with clear structure and hierarchy, and to achieve this through deliberate investment and buy-in from the team.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.