March 2016 Summaries

Confluent has launched Confluent University to provide world-class Apache Kafka training both onsite and at public venues, with courses designed for both operations and development roles. The Operations Training course focuses on installing, monitoring, securing, and managing Kafka clusters, while the Developer Training class teaches how to build applications that publish and consume data from Kafka, both of which are heavily hands-on to ensure practical learning. Initially available in cities like San Francisco, New York City, Redwood City, and Austin, Confluent plans to expand these offerings to more locations in the U.S. and internationally. Founded by the original developers of Apache Kafka, Confluent brings a wealth of experience to its training programs, aiming to provide essential skills for organizations integrating Kafka. In addition, Confluent has been recognized as Microsoft’s 2024 OSS on Azure Global Partner of the Year, underscoring its commitment to delivering superior open source-based solutions on Microsoft Azure. This reflects the growing importance of data streaming as a fundamental component of modern business infrastructure.

The Kafka Streams API simplifies building streaming applications by providing a cluster- and framework-free library that can be used with existing Kafka clusters. This approach reduces operational complexity, making it easier to build and deploy critical core applications and microservices that process real-time data streams. The API integrates tables and streams, allowing for the computation of derived values against table updates using just the stream of changes. It also simplifies windowing, time, and out-of-order events by representing windows as updatable quantities stored in a table. This enables transparent maintenance of derived state, allowing applications to directly query local aggregates. The overall goal is to make stream processing accessible as a mainstream application programming model for asynchronous services, reducing the complexity associated with traditional stream processing frameworks.

The Apache Kafka community had a productive month with several key developments. The release of Apache Kafka 0.9.0.1 and Confluent Platform 2.0.1 brought numerous fixes and improvements, including a patch release for Kafka 0.9. A new release, version 0.10.0, was designated by the community due to significant protocol updates, file format changes, and the introduction of the Kafka Streams processing library. Various KIPs were implemented, reviewed, and merged, such as KIP-41, which allows consumers to limit records returned when polling Kafka, and KIP-32, which adds a timestamp to Kafka's message format. New features like client interceptors and support for delegation tokens are also being discussed. Additionally, the Kafka Connector Hub is live with 12 connectors, and Confluent has released training classes for developers and operations teams. The community also welcomed Microsoft's open-sourcing of their .NET Kafka client.