Improve agent quality with Insights Agent and Multi-turn Evals, now in LangSmith

LangSmith has introduced new capabilities to enhance the monitoring and evaluation of agents in production, focusing on "threads" that represent multi-turn agent interactions. Two new tools, Insights Agent and Multi-turn Evals, have been launched to provide deeper insights into agent performance. Insights Agent categorizes agent usage patterns and identifies failure modes by analyzing production traces, helping teams prioritize improvements based on real user interactions. Multi-turn Evals allow for the assessment of entire conversations to determine if agents meet user goals, moving beyond individual trace evaluations. These tools aim to automate the insight generation process, offering a more comprehensive view of agent behavior and supporting rapid iteration to build reliable AI experiences. Both features are now available to LangSmith users, with further enhancements such as thread-level metrics and dashboards in development.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.