November 2022 Summaries

Application developers often face the challenge of debugging issues that arise only in production environments due to the absence of production data in testing. This guide demonstrates using a service mesh, specifically Kuma, within a Kubernetes cluster to safely mirror production data to development applications, enabling better debugging and production-proof releases. The process involves installing Kuma and related applications, such as Kong Gateway and mockbin.org, to manage ingress traffic and service versions. By configuring Kuma's observability stack and metrics backend with Prometheus and Grafana, developers can track traffic flow and mirror production traffic to a development version without affecting the live environment. This setup allows for enhanced troubleshooting by utilizing production data in a controlled development space, aiding in identifying and resolving issues that are otherwise difficult to reproduce.

Kong is an open-source platform that facilitates the management of APIs and microservices across various environments, such as clouds, data centers, and Kubernetes, by employing intelligent automation. Its solutions support digital innovation by managing the API lifecycle for both legacy and modern architectures, including container runtimes and serverless technologies, thereby enhancing innovation cycles and productivity. Kong Gateway Enterprise has been validated for use on Red Hat OpenShift Service on AWS (ROSA), AWS Graviton processors, and AWS Bottlerocket OS, offering seamless integration and management of APIs and applications in the cloud. The platform leverages the latest AWS Graviton3 processors to provide up to 25% better compute performance, delivering significant cost savings and performance improvements for users. Additionally, the AWS Bottlerocket OS, designed specifically for containerized workloads, provides integrated AWS services, streamlined updates, and reduced management overhead, ensuring a robust operational environment for Kong Gateway deployments.

Kong has been recognized as a Leader in the Gartner Magic Quadrant for Full Life Cycle API Management for the third consecutive year, achieving a position furthest to the right for Completeness of Vision. As a prominent player in cloud-native API management, Kong supports over 560 enterprises with its API life cycle management platform, which includes the world's most adopted API gateway, generating over 350 billion API calls daily. This recognition underscores Kong's dedication to providing a robust and scalable infrastructure for modern applications, as highlighted by its impact on companies like Rush, a Shopify service. Additionally, Kong has received accolades from G2, Forbes, and Inc. Magazine, further affirming its status as a leading private cloud company.

In the realm of Dev Portals, providing users the ability to utilize their own domain is a significant step towards fully customized portals. This functionality was achieved by leveraging the open-source ACME plugin, which integrates with Let's Encrypt to manage TLS certificates dynamically. However, the original plugin had limitations, including a restrictive list of top-level domains and the potential for abuse through the issuance of certificates using IPv4 addresses. To address these, modifications were made to disable the restrictive allow-list and introduce a check against using IPv4 as a certificate common name. Additionally, to prevent malicious misuse by flooding the certificate provider with requests, a custom plugin was developed to verify domain existence in the Konnect portal database before certificate provisioning. This custom plugin runs with a higher priority than the ACME plugin during the TLS connection negotiation phase, ensuring only legitimate domains receive certificates. Consequently, administrators need only add a CNAME in the DNS and enter the custom domain in the portal settings to enable this feature, making custom portals accessible once DNS settings are propagated.

Kong's API Gateway offers critical observability options through Prometheus, an open-source monitoring toolkit, and StatsD, a metric aggregation tool originally developed by Etsy. Prometheus can pull metrics directly from Kong Gateway or use a StatsD exporter to reduce the gateway's workload. Built on OpenResty/Nginx, Kong Gateway utilizes a multi-process, single-threaded architecture that can experience performance issues, such as increased latencies, when the Prometheus plugin with high cardinality metrics is used due to the need for mutex locks on shared memory. To mitigate this, Kong Gateway 3.0 has changed its default settings to not export certain metrics to avoid overhead, although these can be manually enabled if needed. The StatsD plugin is recommended for scenarios with high metric cardinality as it pushes metrics without storing them in memory, thus having minimal impact on performance. In Kong 3.0, advanced StatsD functions have been moved to the StatsD plugin, allowing community users to access previously enterprise-only features, while the StatsD exporter can be deployed as a sidecar for horizontal scaling, avoiding bottlenecks.

Kong Mesh and Kuma 2.0 introduce next-generation policies and eBPF capabilities, offering improved performance and enhanced policy management. This release unifies the versioning of Kong Mesh and Kuma, simplifying customer interactions and upgrades, which can now be seamlessly executed through kumactl or Helm. eBPF integration is designed to optimize traffic flow latency and replace iptables for traffic redirection, utilizing the Merbridge OSS project. The update includes significant UI improvements, such as enhanced navigation, field filtering, and customization in the admin dashboard, along with auditing capabilities for Role-Based Access Control (RBAC) actions. The release also debuts next-gen policies like MeshTrafficPermission, MeshTrafficLog, and MeshTrafficTrace, featuring a new selector mechanism for more granular application. Additional improvements include advanced Datadog integration, TLS configuration options, and increased logging capabilities, all aimed at facilitating more efficient and secure service mesh management.

The blog post discusses how to use Kong Konnect, a SaaS API platform, to implement dynamic request routing based on authentication attributes, specifically JWT token claims. This approach allows routing requests to different paths within the same service address based on user identity, facilitating easier implementation and maintenance. The example provided involves a banking application where users are routed to different banks based on claims in their JWT tokens. By utilizing Kong Konnect, users are relieved from managing ancillary services, allowing them to focus on hosting data planes and business logic. The post also highlights that the same configuration can be applied in self-managed environments and details how to set up and test the routing using tools like decK and a third-party Identity Provider, such as Keycloak, to authenticate users dynamically.

A Solutions Engineer at Kong addresses the common query of transforming Kong logging plugin messages to formats compatible with observability stacks like ELK, Loki, or Splunk by demonstrating how to convert a Kong logging payload to the Elastic Common Schema. The process involves running Kong Gateway in Kubernetes and utilizing two Kong plugins, specifically the serverless pre-function plugin, which operates before other plugins during each phase. The setup requires creating a Kubernetes manifest file and defining KongPlugin resources, where the serverless pre-function plugin logs data into a shared context using Lua code. The transformation is achieved through the custom_fields_by_lua configuration, allowing key-value pairs to modify log fields. This approach facilitates the conversion of Kong log payloads into various formats for easy integration with any observability stack.

The email exchange between Kong's Brad Drysdale and Ahmed Koshok delves into the accelerating trend of cloud adoption in the Asia-Pacific region and beyond, driven by the desire for faster innovation and cost efficiency. They discuss the critical role of APIs and microservices in transforming traditional monolithic applications into agile, cloud-native solutions, emphasizing the necessity of automation throughout the API lifecycle to enhance efficiency, quality, and compliance. Ahmed highlights the importance of addressing the "golden triangle" of quality, time, and cost, noting that automation can help organizations manage these elements effectively while navigating rapid technological shifts. The conversation underscores the need for a strategic approach to API management, advocating for an API-first mindset, design-first creation, and discovery-led adoption to thrive in a modern, distributed IT landscape. Both agree on the transformative power of automation in achieving these goals, with Ahmed suggesting a maturity model for API programs that guides organizations from novice to expert levels.

As infrastructure becomes increasingly distributed, enhancing observability is essential, especially with the rise of microservices architecture. At Kong, the three pillars of observability have been unified to improve the observability platform surrounding Kong Konnect’s infrastructure. Kong Konnect is a comprehensive SaaS API lifecycle management platform tailored for the cloud-native era, featuring Kong Gateway, a highly adopted API gateway. The platform operates with a global control plane in the cloud and a runtime engine within users' networks, embracing microservices from the outset. As Konnect expands with numerous emerging services, building a scalable observability platform has become a priority to enable engineering and CRE teams to swiftly address and prevent issues. Furthermore, as the platform extends its reach to multiple geographies, ensuring robust observability across its entire infrastructure remains critical.

Kong Konnect has introduced several new features aimed at enhancing API security and management, including identity management APIs, improved certificate management, and latency metrics in Kong Analytics. With the new identity management APIs, users can integrate with various identity providers or ERP systems to manage user access, team memberships, and role assignments. The streamlined certificate management feature simplifies operational complexity by offering better visibility and organization of certificates through the Runtime Manager. The public beta release of dynamic client registration in collaboration with Okta allows for more secure client authentication and authorization by integrating directly with identity providers. Service version lifecycle labels have also been introduced, allowing API developers to communicate the lifecycle stage of services to consumers, which is reflected in the Dev Portal. Furthermore, real-time latency monitoring is now available in Kong Analytics, providing insights into API performance to help identify and resolve issues quickly. The Runtime Manager UI has been enhanced for a more intuitive experience, including refresh capabilities and improved navigation. Users are encouraged to explore these features in detail through the documentation or by starting a free trial of Kong Konnect.

Kong's reliance on timers for features like caching and health checks has led to performance issues due to the limitations of OpenResty's timer system, which becomes a bottleneck as environments grow more complex. The introduction of lua-resty-timer-ng, a scalable timer library, aims to address these issues by using a timer wheel algorithm and a timer pool to efficiently manage and schedule a large number of timers while reducing CPU and memory usage. This new library also enhances performance analytics by collecting detailed statistics and generating flame graphs, aiding developers in analyzing timer impacts. Performance tests show that lua-resty-timer-ng outperforms the existing system, especially under heavy timer loads, and it has been incorporated into Kong Gateway 3.0 to boost efficiency. The development team plans to continue refining the library and encourages community contributions to further enhance its capabilities.