Introducing Polly: Your AI Agent Engineer

Polly is an AI-powered assistant integrated into LangSmith that aids in debugging, analyzing, and improving agent performance, addressing challenges encountered by developers when building agents. Unlike simple language model calls, agents involve complex interactions with longer prompts, extended traces, and multiple turns, making it difficult to identify issues manually. Polly excels in this environment by understanding agent architectures, recognizing failure patterns, and offering insights through natural language queries, effectively serving as an expert agent engineer. It assists in debugging individual traces, analyzing entire conversations, and engineering better prompts, streamlining the process of pinpointing inefficiencies and refining agent behavior. Polly's capabilities are powered by LangSmith's comprehensive tracing infrastructure, which captures detailed agent activity, enabling Polly to provide actionable insights and improvements. Now available in beta, Polly represents a significant advancement in agent engineering, promising to enhance the efficiency and reliability of agents developed on the LangSmith platform.