Async Agents: Signal Over Noise

Jessie Frazelle, an experienced engineer, conducted an evaluation of various async AI agents to determine their effectiveness in real production environments, ultimately finding that most agents added more noise than value, often providing irrelevant comments or analyses. Her team, working with a diverse tech stack including Rust, C++, Python, and TypeScript, found that many tools, such as Copilot and Sourcery, frequently made unnecessary or inaccurate contributions to code reviews, sometimes even altering original comments or generating misleading flow diagrams. However, Jessie highlighted Graphite Diamond as a standout agent for its ability to identify significant bugs without contributing unnecessary commentary, which has become a trusted tool in her team's high-stakes projects. Despite her cautious approach to adopting new tools, Jessie appreciates the support these agents provide as a supplementary "second pair of eyes," especially in complex areas where errors can be costly. She concludes that while AI agents are not a replacement for human code review, the right tool, used effectively, can enhance a team's capabilities without overwhelming them with irrelevant information.