AI Gateway is generally available: a unified interface for managing and scaling your generative AI workloads

Cloudflare has announced the General Availability of its AI Gateway during Developer Week in April 2024. The AI Gateway is an AI ops platform that provides a unified interface for managing and scaling generative AI workloads, acting as a proxy between your service and your inference provider(s). It offers features focused on performance, security, reliability, and observability. Key features available today include analytics, real-time logs, caching, rate limiting, support for popular providers, and universal endpoint. The platform aims to provide a single consistent control plane for all AI models and tools, regardless of changes in the rapidly evolving AI space. Future updates will include persistent logs, custom metadata, prompt management tools, built-in eval functionality, feedback API, and fine tune adapters. AI Gateway is available on all plans with core features offered for free, while premium features may be subject to fees.