Reworked Plugin Queues in Kong Gateway 3.3

Kong Gateway's 3.3 release introduces a revamped queue implementation used by various plugins to efficiently handle data transfer to upstream servers, addressing limitations of the previous system. Queues are essential for decoupling data production from submission, preventing high concurrency and resource strain during heavy traffic or when upstream servers face issues. The new design improves memory management by introducing capacity limits, consumer-side batching, and streamlined retry logic with exponential back-off patterns. These queues are memory-based, residing within each worker process and are not shared, ensuring efficient data handling while mitigating performance issues from the previous design, such as extensive timer usage. The update also facilitates better configuration and observability, enabling users to fine-tune queue parameters like max_entries and max_batch_size to align with specific operational needs, while automatic conversions assist in transitioning from previous configurations. This new queue system is integrated with plugins like Datadog, HTTP Log, and OpenTelemetry, with plans to extend its use across more plugins in the future.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.