Company
Date Published
Author
Claudio Acquaviva
Word count
1161
Language
English
Hacker News points
None

Summary

In February 2024, Kong introduced the first dedicated AI gateway, providing a robust platform to enhance performance, observability, and policy enforcement for Generative AI (GenAI) workloads. The Kong AI Gateway offers a universal API to centrally secure and govern traffic to large language models (LLMs), AI agents, and MCP servers, helping organizations manage soaring AI costs with techniques like token rate limiting and response caching. Kong's infrastructure is built on its established API platform, ensuring performance and scalability for demanding AI projects. Benchmark tests conducted on AWS, comparing Kong AI Gateway with Portkey and LiteLLM, revealed that Kong significantly outperformed its competitors in terms of requests per second and latency. The tests showed Kong Konnect Data Planes being over 228% faster than Portkey and 859% faster than LiteLLM, with considerably lower latency. This comprehensive platform allows organizations to unify API and AI management, offering control, visibility, and scalability across diverse deployment environments, whether they are monolithic, microservices-based, or multi-cloud.