Testing With Live Agent Runtimes in Conductor

Building on mocked workflow-routing tests, this tutorial shows how to use Conductor’s live agent runtime and evaluation tools to test a three-agent content pipeline under real execution conditions. It introduces pytest fixtures for a reusable runtime, a streaming wrapper that captures agent handoff events, and `EvalCase` and `CorrectnessEval` definitions that verify agent order, topic preservation, expected tool use, output content, and limits on excessive tool calls. The guide demonstrates how live evaluations expose failures that mocks can miss, including a deliberately broken pipeline missing its summary agent, while Conductor records each execution for inspection in its UI. It also covers isolated happy-path tests for search, summary, and editor agents, unhappy-path tests that confirm evaluator failures are reported for missing expected tools, and parallel test execution with pytest-xdist. The recommended approach is to retain fast deterministic mocks for routing checks while using slower, token-consuming live evaluations before releases to detect behavioral changes in actual agents and tools.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.