Agent traffic is 62% cache reads, and it should change how you think about routing

For coding-agent workloads, LLM routing should account for session-level prompt caching rather than treating each request independently, because 62.1% of the 46 billion input tokens handled by the Weave router over 30 days were discounted cache reads. Since provider caches are tied to a model and an exact conversation prefix, moving a session to another model requires resending its accumulated context at full input cost, often outweighing apparent per-request savings. Weave therefore makes a substantial routing choice on the first turn and pins later turns to that model by default, with 14,316 pinned sessions averaging 10.4 turns, 33.6% of requests served through pins, and only 1.4% of pinned sessions switching models. Low failover rates and frequent agreement between pinned models and shadow routing choices suggest that most initial selections remain appropriate, although switching can still be worthwhile for provider failures, repeated agent errors, or a shift to more complex work. The approach argues that chat traffic may suit request-by-request routing, while agent systems need to price cache loss explicitly and use deliberate session un-pinning decisions.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.