Agent cost management is about more than the model

Agentic applications can incur rapidly growing LLM costs because individual requests often trigger multiple model calls and large tool outputs remain in accumulated context for later steps, making observability and context management essential for scaling. Arize AX introduces a managed Cost Agent that analyzes tracing data, identifies high-spend spans, correlates cost with quality, and, when connected to GitHub, can propose code changes through pull requests while leaving deployment decisions to engineers. In an example involving a LangGraph financial assistant, the agent found that a web-page retrieval tool returned excessively large payloads that were repeatedly resent to the model, alongside missed prompt-caching opportunities, unnecessary conversation-history ingestion, and oversized financial-data responses. It proposed six changes—including payload truncation, caching, message filtering, reduced tool limits, and context trimming—estimated to cut the monthly bill by 35–40%, with measurable post-deployment predictions. AX provides per-span and per-trace LLM cost tracking, while its Cost Agent, available in Enterprise beta, can run once or on a recurring schedule to continuously audit and recommend optimizations.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.