How to test agent tool descriptions with Harbor and Phoenix

A benchmark using Harbor and Arize Phoenix found that changing the description of a skill-loading tool eliminated 51 unnecessary Codex skill loads, reduced benchmark cost from $11.85 to $8.34, and preserved perfect correctness. The evaluation used 18 fixed, sandboxed tasks with reference solutions and verifiers that judged final answers rather than requiring specific tool-use paths, allowing the same tasks to compare PXI, Codex, and Claude Code configurations. Running each task three times per agent exposed that agents with identical pass rates could differ substantially in tool calls, tokens, and costs: Codex frequently loaded an irrelevant error-analysis skill, while Claude Code did not. Phoenix connected verifier scores with execution traces and diagnostics such as tool counts, making the inefficiency visible. The walkthrough emphasizes validating LLM judges carefully, including guidance for equivalent answer names, establishing a stable baseline, changing only one variable, and rerunning the identical benchmark to measure whether efficiency improvements maintain correctness.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.