Agent tool selection: The tool your agent almost called

Arize describes an approach to agent tool selection that separates choosing a tool from generating its arguments: a decision model assigns probabilities and an overall confidence score to available tools, while a chat model executes the selected tool call. In tests of a five-tool shopping agent across 100 messages, low router confidence correlated with higher error rates, with about one-third of selections below 0.7 judged wrong compared with fewer than one in ten above 0.8. Using a 0.7 confidence gate, where low-confidence choices revert to the chat model’s native tool selection, reduced empty or unfinished responses from 21 to 3 compared with always enforcing the router’s choice, while maintaining similar judged correctness. The post explains how Phoenix DECISION spans, annotations, datasets, experiments, LLM-based judging, and code-based expected-tool checks can help teams trace decisions, diagnose whether failures stem from selection or execution, and establish thresholds from their own data. It also notes that thresholds are agent-specific and suggests the approach may be increasingly useful for agents with many overlapping tools, where routing could reduce the tools shown to the chat model.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.