Prompt caching benchmark: high cache reuse doesn’t always mean lower cost

Prompt caching can reduce repeated-input costs in multi-turn agents by reusing instructions, context, and conversation history, but benchmark results using a 20-conversation e-commerce assistant workload showed that high cache reuse alone does not guarantee lower overall cost. Harbor and Arize Phoenix compared four model/provider paths across repeated conversations of 5 to 20 turns, finding that DeepSeek had the highest aggregate cache-read rate at 93.6%, the lowest estimated cost at $1.17 per 100 runs, and the fastest average latency at 34.7 seconds, while Claude had an 89.8% cache-read rate but the highest estimated cost of $14.63 because output tokens accounted for much of its spending. Cache reuse increased with conversation length for all models, with GPT showing the largest increase from 34.2% at five turns to 87.0% at 20 turns, whereas DeepSeek began with high reuse even in shorter conversations. The comparison reflects combined model and provider behavior rather than caching efficiency in isolation, and Phoenix’s experiment and trace views allow teams to inspect cache reads, token usage, estimated costs, latency, and individual LLM calls. The results emphasize that organizations should evaluate caching on their own workloads while considering output volume, pricing, latency, and task quality alongside cache-read rates.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.