December 2023 Summaries

Confluent has introduced a new feature called Confluent Access Management that allows organization admins to manage users and user permissions within the Azure portal after creating a Confluent organization. This streamlines the process of accessing Apache Kafka resources on Confluent, providing advanced controls for managing user access and permissions. The feature supports role-based access control (RBAC) and aims to simplify user management while enhancing security by centralizing user management in the Azure portal.

The cruise industry is facing challenges in providing a seamless customer experience due to legacy systems that struggle with real-time data streaming. The introduction of microservices, NoSQL databases, and event-driven architecture has brought new capabilities to onboard applications, but ship-to-shore communication remains a challenge. Data streaming unlocks new capabilities for customer experience across all industries, including the cruise industry. Confluent's cloud-native platform provides a central nervous system for reservations across an entire fleet of ships and back at shore, enabling real-time data streaming and synchronization between ship and shore clusters. By leveraging Confluent's technology, cruise liners can provide a modern suite of customer-facing applications onboard, cultivate real-time insights, and serve their customers better, providing a consistent experience for cruisers regardless of which system they use to create reservations.

In February 2021, McAfee embarked on a replatforming strategy to transition from a monolithic architecture to a cloud-native event-driven microservices architecture, aiming to enhance scalability, fault isolation, and time to market while reducing costs. This strategic shift was driven by the need to overcome the limitations of their existing infrastructure, which impeded their ability to rapidly innovate and protect over 500 million users from cyber threats. By adopting microservices, McAfee sought to improve business velocity and system availability, emphasizing the importance of decoupling services to ensure resilience and real-time responsiveness to security events. The transition process involved several phases, including modernizing the technology stack, leveraging managed services like Confluent Cloud for efficient event management, and emphasizing the importance of securing organizational buy-in by demonstrating immediate business benefits. This transformation not only enhanced McAfee's operational efficiency but also shifted their focus from device-centric to user-centric protection, aligning their architecture with contemporary technological standards to support future growth and innovation.

Confluent's serverless Apache Flink service abstracts away the underlying infrastructure and provides a fully managed environment for stream processing applications, empowering users to focus on app development without operational overhead. The service offers elastic autoscaling with scale-to-zero, evergreen runtime and APIs, and usage-based billing, reducing infrastructure costs and providing accurate billing based on fluctuating usage. The serverless architecture is designed to pass dynamic billing features and benefits on to users, making mixed workloads, including variable and short-lived explorative workloads, very cost-effective. The service provides fast rescaling with sub-second rescaling, minimizing downtime duration driven by how much time it takes to repartition the state into the new degree of parallelization before restarting the Flink job. With a base price of $0 for compute pools and no additional charges for state management or networking, users can take advantage of accurate usage-based billing and efficient resource allocation, resulting in cost savings.

A telecom company in Spain has revolutionized its operational efficiency by adopting a real-time, event-driven architecture powered by Confluent Cloud. This transformation addresses several challenges associated with relying on third-party service providers for customer network installations, such as customer misperceptions, accountability issues, and quality control. The new system leverages microservices, HTTP sink connectors, and stream processing to provide immediate updates on service tasks, enhancing collaboration and communication with third-party companies. This approach has eliminated manual workflow inefficiencies, increased scalability, and improved customer satisfaction by ensuring timely task assignments and reducing errors. The implementation has led to significant operational benefits, including incident-free operations, improved collaboration, enhanced efficiency, and revenue growth, positioning the company to adapt swiftly to evolving business needs and maintain a competitive edge in the telecom industry.

Data streaming platforms are becoming increasingly important in today's fast-paced world, powering workflows across various industries. Confluent, a leader in The Forrester Wave™: Streaming Data Platforms, Q4 2023, is at the heart of this revolution. Built by the founders of Apache Kafka®, Confluent offers a cloud-native, complete data streaming platform for every use case. It helps transform complex batch systems into real-time experiences faster, safer, and more cost-effectively than ever. The company's commitment to delivering outstanding open source-based applications and infrastructure solutions on Microsoft Azure has earned it the recognition as Microsoft's 2024 OSS on Azure Global Partner of the Year.

Apache Kafka has become a standard for streaming data, but its proliferation across enterprises often leads to increased complexity and cost due to siloed deployments by independent teams. This blog post from Confluent addresses the challenges faced by a global enterprise with over 40 Kafka clusters, highlighting issues such as lack of common strategy, repetitive DevOps work, and operational inefficiencies. By adopting a centralized data streaming strategy, the enterprise reduced its Kafka DevOps workforce from 50 to 15, achieving significant cost savings and enhanced focus on business problems. The consolidation of Kafka clusters also led to considerable infrastructure cost reductions while maintaining high reliability and performance. The post discusses the benefits and trade-offs of centralization versus decentralization, emphasizing the importance of aligning cluster strategies with data strategies while considering factors like cost, value, business needs, and technical requirements. Confluent suggests that a centralized platform can improve service quality, standardize processes, and mitigate risks, but the optimal strategy depends on the specific needs and structure of each organization.

The text discusses how Confluent and Apache Kafka® can be used in the engine manufacturing industry to provide real-time visibility during critical component machining, assembly, and testing processes. By bringing IT into the traditional manufacturing world of operational technology (OT), product quality is increased, waste is reduced, and risk is minimized. The article also highlights how data infrastructure transformation is crucial for modern manufacturers, as catching issues with specific components in real-time can dramatically reduce costs associated with scrap, rework, and brand damage. Furthermore, predictive maintenance leverages streaming data to identify issues before they occur, enabling proactive remediation and improving equipment lifespan. Confluent Cloud is a fully managed data streaming platform that supports large-scale durable data streaming use cases in the manufacturing space.

The latest development in Confluent Cloud is the availability of Data Portal, a developer-friendly, self-service UI that provides an easy and curated way for users to find, understand, and enrich their Kafka data streams. This tool leverages Stream Governance capabilities to empower data users to interact with their organization's Kafka data streams safely, efficiently, and collaboratively. With Data Portal, users can search and discover existing topics, request access to topics through an approval workflow, set up clients and query data with Apache Flink, and streamline access request management. The tool aims to simplify data governance initiatives by providing a user-friendly interface for managing the availability, integrity, and security of data used across an organization.

Confluent Cloud has released Data Portal, a developer-friendly, self-service UI that provides an easy and curated way for users to find, understand, and enrich all of their Kafka data streams. With Data Portal, users can search and discover existing topics across the organization, request access to topics through an approval workflow, set up clients and query data with Apache Flink, and gain a unified view of all available Kafka topics by environment. The platform aims to streamline access request management, eliminate manual permission assignments, and ensure security and access controls remain in place, while also providing an enhanced user experience for stream governance initiatives.