May 2025 Summaries

Kong has announced the general availability of AI Manager within Kong Konnect, a platform designed to manage API, AI, and event connectivity across digital applications and AI agents. The new AI Manager capabilities allow users to expose, govern, secure, and observe large language model (LLM) traffic using a user-friendly interface. These features complement existing API and declarative configurations, enabling comprehensive management of LLM and MCP traffic via Kong’s AI Gateway technology. AI Manager provides tools to manage AI policies, curate LLM catalogs, visualize agentic maps, and observe LLM analytics, offering insights into token, cost, and request consumption. It supports both hybrid AI Gateway deployments and managed Dedicated Cloud Gateways, with native Kubernetes support through the Ingress Controller or Operator. Users can also access over 50 specialized AI capabilities, including federated governance and PII sanitization, enhancing the ease of managing AI infrastructure across major cloud platforms.

As businesses increasingly integrate artificial intelligence (AI) and large language models (LLMs) into their operations, they face the challenge of managing a surge in AI-related traffic, which can lead to unpredictable costs, latency issues, and reliability concerns. To address these challenges, AI gateways serve as sophisticated intermediaries that regulate traffic flow, optimize costs, and maintain system stability. These gateways perform critical functions such as traffic routing, rate limiting, caching, model fallback, load balancing, and observability, ensuring efficient and cost-effective management of AI resources. Rate limiting prevents system overload by controlling request flow, while caching reduces latency and costs by storing frequently accessed responses. Model fallback and intelligent retry mechanisms maintain service continuity during disruptions, and load balancing distributes workloads to avoid overburdening any single model or provider. Advanced load-balancing algorithms and adaptive management strategies further optimize resource allocation and performance, allowing businesses to dynamically respond to traffic spikes. Solutions like Kong Gateway offer built-in features to implement these strategies, helping organizations transform chaotic AI traffic into a streamlined and efficient system, thereby enhancing user satisfaction and cost control.

Kong has expanded its headquarters to 44 Montgomery Street in San Francisco, providing more space and amenities like a 24-hour gym and outdoor social areas for its employees. The new office supports both remote and hybrid work environments while emphasizing the importance of in-person collaboration for team building and client engagement, backed by a study indicating a 27% increase in business sales through teamwork initiatives. Partnering with Gable, Kong offers employees access to a variety of global coworking spaces, signifying its growth strategy, which includes a recent $175 million Series E funding round that brought its valuation to $2 billion. With over 700 employees across 25 locations, Kong continues to focus on innovation, particularly with updates to its Kong Konnect platform, Kong AI Gateway, and the launch of a new Kong Event Gateway. Its commitment to employee satisfaction has earned it a Great Place to Work® certification, highlighting a culture of inclusivity and support, while also inviting prospective employees to explore career opportunities within the company.

Agentic AI is transforming the landscape of API development and management by introducing autonomous, goal-oriented agents that can interpret context, make adaptive decisions, and drive outcomes in real-time, marking a significant shift from traditional static AI models. This advancement allows APIs to deliver dynamic, contextual, and personalized responses, optimizing performance, security, and scalability, and automating complex workflows to free developers for higher-level tasks. Integrating agentic AI into APIs offers numerous benefits, including enhanced efficiency, improved developer experience, and better scalability, while necessitating a reevaluation of API strategies and governance to address ethical, security, and accountability concerns. The competitive landscape is shaped by leading cloud providers, innovative startups, and pioneering organizations that are pushing the boundaries of AI-driven APIs, demonstrating the potential for new applications and long-term innovation in API ecosystems. To capitalize on the opportunities presented by agentic AI, businesses must invest in AI-centric infrastructure, cultivate an AI-fluent workforce, and foster a culture of innovation and agility to maintain a competitive edge in the rapidly advancing digital landscape.

The guide provides an in-depth exploration of strategies to secure Large Language Models (LLMs) against emerging threats, focusing on the OWASP LLM Top 10 vulnerabilities, such as injection attacks, data leaks, and model theft. With the rapid adoption of LLMs across industries, the guide emphasizes the necessity for AI developers, product managers, security leads, and compliance officers to implement robust security frameworks. It highlights real-world examples like the DeepQuery breach to illustrate risks, describes the unique challenges posed by LLMs compared to traditional web applications, and outlines defense strategies including input validation, encryption, access control, and regular security testing. The guide also covers compliance with evolving regulations like GDPR and CCPA, emphasizing the importance of building a secure AI ecosystem through principles like least privilege, role-based access control, and ethical AI guidelines. As LLMs become integral to business operations, securing them is vital to safeguarding an organization's financial health, reputation, and competitive edge.

Kong has launched the Kong Event Gateway, which extends its API platform to support event-driven architectures, particularly focusing on real-time data and event streaming with Apache Kafka. This new gateway allows organizations to manage, secure, and expose Kafka-based event streams in a manner similar to traditional APIs, streamlining access and control with a unified platform. It supports two architectural patterns: Protocol Mediation, which facilitates access to Kafka over HTTP, and the Kong Native Event Proxy, offering policy-based controls using native Kafka protocols. The Event Gateway aims to reduce operational complexity by centralizing security, access management, and monitoring, ultimately enhancing the developer experience and enabling faster innovation. Additionally, it introduces features like Virtual Clusters and Virtual Topics to optimize infrastructure use and promote multi-tenancy within Kafka environments. Future updates will include support for more broker platforms, expanding the gateway's capabilities beyond Kafka.

Anthropic's Model Context Protocol (MCP) is an open standard designed to provide AI models, particularly large language models (LLMs), with the necessary context—such as external data, tools, and services—to perform tasks effectively. Released in 2024, MCP aims to standardize how LLMs access additional context, thus eliminating the need for fragmented custom integrations. While some view MCP as a mere repackaging of tool calling or APIs, it introduces interoperability and intention-based communication, allowing LLMs to reason more effectively about when and why to use certain tools. MCP enables agentic workflows by providing tools, memory, and prompts that let LLMs make autonomous decisions. However, transitioning to remote MCP servers introduces challenges such as secure authentication, high availability, and the risk of tool poisoning attacks. API gateways can help manage these complexities by providing centralized security, load balancing, and developer onboarding. As organizations like Kong apply these principles to MCP, the protocol is poised to become a foundational component in AI-driven workflows.

As AI-driven applications become increasingly integral to business operations, managing their usage and associated costs is crucial. Large language models (LLMs) from providers like OpenAI, Google, Anthropic, and Mistral can lead to significant expenses if not properly governed. Kong’s AI Gateway offers solutions by implementing token rate-limiting and tiered access features, allowing organizations to control AI usage and prevent overuse. Token management is vital as it directly impacts costs, with tokens representing segments of text that scale with query complexity. Through tiered access control, businesses can allocate resources efficiently, ensuring premium AI resources are reserved for high-tier users while maintaining performance and cost-effectiveness. Kong’s approach, which includes plugins like AI Rate Limiting Advanced, integrates AI-specific token logic into traditional API management workflows, enabling centralized control over AI resources. This system not only safeguards against misuse and overload but also aligns with compliance and governance standards, providing a strategic framework for AI resource management.

Cross-charging models in enterprise platforms, which involve reallocating costs from central IT functions to various lines of business (LOBs), are often met with skepticism due to their complexity and potential drawbacks. These models aim to recover operating and hosting costs by billing internal consumers or producers, distinct from product monetization. The process requires a detailed understanding of fixed and variable costs, such as software licenses and cloud infrastructure, and decisions regarding whether charges should be based on usage or consumption. Although cross-charging can make teams more cost-conscious, it may also discourage platform adoption, disrupt reuse of resources like APIs, and lead to defensive behavior or shadow IT practices. Building a comprehensive cross-charging model demands significant effort and resources, potentially offsetting any financial benefits. Therefore, a simplified approach is recommended, leveraging tools such as Kong Konnect's analytics to maintain cost-effectiveness and minimize friction in value streams.