July 2018 Summaries

Confluent Platform 5.0, built on Apache Kafka 2.0, introduces several enhancements aimed at streamlining enterprise streaming infrastructure and application development. Key features include improved security with an LDAP authorizer plugin, faster disaster recovery through automatic offset translation in Confluent Replicator, and enhanced monitoring capabilities in Confluent Control Center, such as broker configuration views and consumer lag insights. For developers, the platform offers advanced tools like a new graphical user interface for KSQL, support for nested data, user-defined functions, and comprehensive join capabilities, facilitating more powerful data processing and application building. Additionally, the new Confluent MQTT Proxy simplifies IoT integration by eliminating the need for third-party MQTT brokers, enabling direct communication with Kafka. Overall, Confluent Platform 5.0 aims to make Kafka easier to manage while enhancing its functionality for both operators and developers.

Kafka Summit San Francisco is a premier event for developers, DevOps professionals, software architects, and anyone building with Kafka, offering a platform to share experiences, plan the future of streaming technologies, and learn about business cases. The 2018 summit features over 50 sessions across multiple tracks, including Internals, Pipelines, Streams, and The Business Case, with highlights such as tech leaders and industry giants sharing their knowledge on Apache Kafka and Apache Flink. Early bird pricing ends August 1 to save $200 off the standard conference pass, with registration now open for attendees to secure their spot at this event focused on data streaming and cloud services.

Confluent has introduced its first set of Ansible playbooks designed to streamline the setup and installation of the Confluent Platform, making it easier for engineers to create a production-ready proof-of-concept cluster quickly. These playbooks facilitate the deployment of Confluent Platform services with systemd service unit files, simplifying the setup with default usernames and log locations for consistency and ease of use. They offer optional automated authentication and encryption setups, supporting modes like PLAINTEXT, SSL, and SASL_SSL, to help users demonstrate the security capabilities of the platform efficiently. The playbooks allow for the comprehensive deployment of all platform components, including Apache ZooKeeper, Kafka brokers, and Confluent Control Center, enabling users to manage topics and connectors directly from the UI. Confluent aims to ensure the success of users' installations, whether they are initiating a proof-of-concept or preparing a full production deployment, and invites feedback and contributions to the open repository.

The text discusses the evolving landscape of enterprise integration solutions, focusing on the limitations of traditional systems like enterprise service buses (ESBs) and extract-transform-load (ETL) tools, which often lead to tightly coupled architectures. It highlights the shift towards using Apache Kafka, an open-source streaming platform, as a more flexible and scalable solution for integrating legacy and modern applications. Kafka's distributed architecture enables real decoupling, allowing systems to publish and consume data independently in real-time or batch processes. The text advises against replicating ESB-like patterns around Kafka to avoid inflexibility and central dependencies, promoting instead the use of Kafka's ecosystem for building event-driven infrastructures. It also emphasizes the importance of gradual integration with existing legacy systems using tools like ETL and ESB, which can complement Kafka's capabilities. Additionally, it mentions the release of Apache Kafka 3.8.0 and the availability of managed services like Confluent Cloud to simplify adoption.

The June 2018 preview release of Confluent Platform introduces several enhancements, including full support for Avro data in Control Center's topic inspection feature and the addition of feature access toggles within Confluent Control Center. Additionally, KSQL now supports complex nested data types with STRUCT data type, user-defined functions (UDFs), and user-defined aggregate functions (UDAFs). Furthermore, this release introduces Stream-Stream and Table-Table joins to cover all available join operations in Kafka Streams.