Response Caching: Zero Cost for Identical Requests

Brian Thomas introduces a new response caching feature that allows users to cache identical requests made to the OpenRouter platform, leading to faster response times and zero token billing for repeated calls. When caching is enabled by adding a specific header or setting a preset configuration, OpenRouter generates a cache key by hashing the request details, enabling the immediate return of cached responses if identical requests have been made previously. This feature, which supports both streaming and non-streaming requests across various data types, significantly reduces response times from seconds to milliseconds without affecting the client code. While standard-sized requests are cacheable, very large multimodal payloads are not eligible. The caching mechanism is independent of prompt caching and skips the provider entirely, making it particularly beneficial for agent retries, test suites, and repeated context processing. The cache is specific to each API key, ensuring various endpoints like chat completions, responses, messages, and embeddings can utilize it, although some legacy endpoints are not supported yet. The feature is currently in beta and being monitored for performance, while cache activity is easily trackable through the Activity log.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.