Agent tracing: how to trace and debug AI agents in production

Agent tracing provides an in-depth view of how a support agent's execution path unfolds during a run, capturing each step's inputs, outputs, and the sequence of decisions leading to the final response. This method proves essential in debugging production failures, such as when a retrieval step returns an outdated policy document that leads to incorrect answers. Unlike traditional logging or LLM tracing, agent tracing offers a structured parent-child span tree that connects model calls, tool calls, retrievals, and state changes, providing comprehensive visibility into the agent's decision-making process. This detailed trace enables teams to pinpoint the exact step causing a failure, facilitating efficient debugging and forming the basis for creating regression evaluation cases to prevent future errors. Braintrust's agent tracing helps transform production failures into permanent evaluation cases, allowing teams to test changes to prompts, models, retrievals, or tools before they reach production, thereby enhancing the reliability and accuracy of AI-driven agents.