How an in-house review bot cut our PR merge time in half

Vapi engineer Lat Ware developed the Pedantic Review Bot after observing that pull requests took more than 130 hours on average to merge, allowing the company to use slower, more thorough AI reviews than commercial tools; after deployment across all pull requests, average merge time fell to 65 hours. The bot analyzes code, runs tests or reproductions before commenting, assigns severity levels, avoids repeat reviews, and is separated into an untrusted analysis process and a limited posting process, while a companion architecture bot identifies higher-level design issues. From September 2 to October 2, 2026, the main bot reviewed 1,694 pull requests and identified 363 blockers, many involving build, testing, CI, and deployment failures that normal checks could miss. Experiments with nearly 200 prompts found that a concise instruction emphasizing aggressive, actionable inline reviews outperformed more elaborate approaches when false positives were properly penalized, while repeated lower-cost runs generally detected more issues than a single maximum-effort run. Ware also found that reviewers need explicit guidance on whether to flag pre-existing defects, and notes that broader testing, seeded defects, and tracking of engineer responses are still needed to measure the bots’ effectiveness more reliably.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.