Stories from the Factory Floor: Building a self-driving ops triage loop

LaunchDarkly’s Foundation team developed a three-agent incident-response workflow that investigates Datadog and Spinnaker alerts, validates remediation plans, and opens scoped pull requests while keeping humans involved for review and uncertain cases. The agents communicate through Jira rather than Slack, after unreliable reaction-based triggers showed that durable, auditable ticket states and automations were more reliable for handoffs, retries, and tracking. Triage and validation use more capable reasoning models to assess incidents and challenge proposed fixes, while a less expensive implementation model executes approved, well-defined changes. The system uses custom cloud MCP connections for tools such as Datadog and Courier, applies deterministic deduplication keys to prevent repeated tickets and PRs, and distinguishes completed fixes, duplicates, deliberate rejections, and cases awaiting human input. Remaining limitations include incomplete access to CI test artifacts, repository-scope restrictions, and the difficulty of distinguishing flaky test failures from real defects, while a longer-term goal is to assess error spikes before they generate pages.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.