LangSmith Engine v2: Red teaming and automated testing

LangSmith Engine v2 expands the platform’s in-product agent for diagnosing and fixing AI agent issues, building on its analysis of more than 70 million traces and tens of thousands of identified problems since its May launch. It scans production traces to group related failures, identify root causes, propose fixes, generate evaluation examples, and monitor outcomes, while new red-teaming capabilities proactively test agents for weaknesses such as hallucinations and system-prompt violations before they affect users. The release also identifies less obvious production problems, including inefficient tool-use trajectories and negative trends in error rates, latency, and cost. For LangSmith Deployment users in private beta, Engine can reproduce failures, iteratively test and refine proposed fixes, and provide validated changes that can be opened as pull requests for deployment. LangSmith reports that recent improvements have more than doubled issue-detection performance and improved fix effectiveness by 25%, while upcoming self-hosted support will add bring-your-own-key inference to keep trace data within customers’ virtual private clouds.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.