Introducing LangSmith Engine

LangSmith Engine is a newly launched tool designed to automate the process of improving agents by analyzing production traces, identifying patterns in failures, diagnosing root causes, and proposing fixes to prevent regressions. By clustering failures into named issues and suggesting targeted code changes, LangSmith Engine simplifies the traditionally manual agent development cycle, which involves tracing, identifying patterns, and creating ground truth datasets. The Engine proposes automated actions such as drafting pull requests, creating custom online evaluators, and adding examples to offline evaluation suites, thereby enhancing eval coverage and reducing the need for manual intervention. With its ability to monitor explicit errors, trace anomalies, and negative user feedback, LangSmith Engine aims to streamline issue resolution and improve agent robustness. It is currently available in public beta, with companies like Cogent, Harmonic, and Campfire already utilizing it to efficiently address issues and minimize triage time.