ClickGap: Autonomous QA for ClickHouse

ClickGap is an autonomous QA agent developed by ClickHouse to review merged pull requests in its performance-critical C++ database, where high merge volume and extensive existing CI still leave gaps involving untested behavior, regressions, and performance losses. In its first five months, the bot filed roughly 500 issues and opened about 200 test-coverage pull requests, with many findings resolved through linked fixes, after reproducing defects such as unintended semantic changes, memory-spill failures, ineffective skip indexes, and query slowdowns. The system validates findings through real builds, executed reproducers, coverage mutation tests, adversarial model review, deterministic checks, and ten evidence gates before it can create a public issue or pull request. It also bisects regressions, tests affected supported releases, and avoids attributing defects to a pull request unless the evidence supports a causal connection. ClickGap’s design emphasizes maintainer trust, incorporating safeguards against spam, false positives, flawed CI interpretation, excessive filing, and repeated mistakes, while using human feedback to update a curated memory system. The project argues that specialized knowledge, measurable validation, deterministic automation, concise reports, and continuous outcome-based evaluation are more important than simply making an AI agent appear capable.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.