AI Gateway Benchmark: Kong AI Gateway, Portkey, and LiteLLM
Blog post from Kong
In February 2024, Kong introduced the first dedicated AI gateway, providing a robust platform to enhance performance, observability, and policy enforcement for Generative AI (GenAI) workloads. The Kong AI Gateway offers a universal API to centrally secure and govern traffic to large language models (LLMs), AI agents, and MCP servers, helping organizations manage soaring AI costs with techniques like token rate limiting and response caching. Kong's infrastructure is built on its established API platform, ensuring performance and scalability for demanding AI projects. Benchmark tests conducted on AWS, comparing Kong AI Gateway with Portkey and LiteLLM, revealed that Kong significantly outperformed its competitors in terms of requests per second and latency. The tests showed Kong Konnect Data Planes being over 228% faster than Portkey and 859% faster than LiteLLM, with considerably lower latency. This comprehensive platform allows organizations to unify API and AI management, offering control, visibility, and scalability across diverse deployment environments, whether they are monolithic, microservices-based, or multi-cloud.
| Trend | Post Mentions | Total Month Mentions | Posts | Companies | MoM |
|---|---|---|---|---|---|
| LLM | 6 | 4,152 | 612 | 181 | +19% |
| Kubernetes | 2 | 1,602 | 228 | 83 | -1% |
| Observability | 2 | 2,058 | 407 | 126 | +10% |
| AI Agents | 1 | 2,211 | 458 | 158 | +26% |
| MCP | 1 | 3,238 | 234 | 106 | +32% |
| Serverless | 1 | 889 | 215 | 78 | +28% |
| Vector Search | 1 | 1,836 | 305 | 108 | +20% |
Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.