Prompt Caching is GA on DigitalOcean Serverless Inference: Up to 90% Off the Context Your Agents Resend

DigitalOcean has made prompt caching generally available for its Serverless Inference platform across supported open models, automatically discounting cached input tokens by up to 90% and reducing time to first token by up to 80%, depending on the model and prompt. The feature addresses the repeated context sent by AI agents, such as system prompts, tool definitions, files, and prior conversation history, through cache-affinity routing that directs requests to replicas already holding the relevant prompt cache rather than relying on random load balancing. DigitalOcean reports that internal October 2026 tests increased cache hit rates from roughly the mid-30% range to more than 90% for flagship models, with reported results varying by model and prompt length. The release also introduces the optional prompt_cache_key parameter to isolate cache use among teams or customers within an account, extends support to the Responses API, increases context windows to 256,000 tokens or more, and adds a dashboard showing cache hit rates, model-level performance, and estimated savings.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.