Queues for Apache Kafka® Is Here: Your Guide to Getting Started in Confluent

Queues for Kafka, now generally available on Confluent Cloud and soon to be on Confluent Platform with the Apache Kafka 4.2 release, introduces queue semantics directly into Kafka, addressing the challenges of maintaining separate messaging systems for streaming and task distribution. This innovation allows organizations to consolidate their messaging infrastructure, reducing operational overhead and total cost of ownership while maintaining the benefits of both traditional queues and Kafka. Key features include the introduction of share consumer APIs and share groups that enable elastic scaling beyond partition limits and per-message processing controls, such as acknowledgment, release, and renewal of messages. These capabilities allow Kafka to handle both streaming and queuing workloads on a single platform, leveraging Kafka's durability and scalability without the constraints of traditional consumer groups. The release is available on both Confluent Cloud, offering a fully managed experience with advanced monitoring and management tools, and Confluent Platform, which provides deep integration with UI visibility and advanced tuning options. The choice between using share consumer API or traditional consumer API depends on workload requirements, with the former catering to operational tasks requiring elastic scaling and the latter to analytical tasks requiring strict ordering.