What agent traces can tell you without an LLM judge

tracelint is an open-source linter for OpenInference agent traces, including traces collected by Arize Phoenix, designed to detect structural agent failures that can be proven deterministically rather than judged semantically by an LLM. It distinguishes hard defects, such as invalid tool arguments, nonexistent tool calls, or reuse of a declared failed result in a side-effecting action, from candidate issues like repetitive calls that may be legitimate retries, while marking insufficiently evidenced cases as not checked. A release-agent example shows how an agent deployed a Jenkins build marked UNSTABLE despite a policy to deploy only successful pipelines; by declaring in a tools.json file that UNSTABLE represents failure and that deployment has side effects, tracelint can identify the trace as a hard defect and return a CI-failing exit code. The tool can generate initial tool definitions from traces, integrate with pytest and GitHub Actions, and complements rather than replaces agent evaluations, which remain necessary for assessing task-specific correctness, strategy, grounding, and policy compliance.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.