June 2024 Summaries

Redpanda Connect with Ockam introduces the first zero-trust streaming data platform, offering a secure, scalable, and easy-to-deploy solution for enterprises needing robust data streaming capabilities. This collaboration leverages both companies' open-source foundations to simplify the construction of secure streaming data connections, eliminating the complexities typically associated with enterprise Kafka product vendors. Redpanda's declarative platform allows developers to create simple, stateless processing steps, while Ockam's cryptographic protocols ensure data integrity and prevent risks such as MITM and supply chain attacks. The platform provides end-to-end encryption, enabling businesses to confidently transmit sensitive data and develop high-value applications previously hindered by privacy and governance concerns. With over 250 pre-built integrations, Redpanda Connect allows seamless connection across various applications and infrastructure, while Ockam simplifies the management of secure channels through automatic key handling. Redpanda Cloud offers a fully managed service, ensuring continuous monitoring and maintenance to meet strict performance and security requirements.

Redpanda Connect is a lightweight stream processor designed to facilitate the creation of streaming data pipelines using a declarative approach with YAML, offering a streamlined alternative or complement to traditional data engineering tools like enterprise integration frameworks, log aggregators, and ETL engines. Written in Go and deployable as a static binary without external dependencies, it features an extensive ecosystem of over 280 high-performance connectors for diverse use cases, including message format transformation, code execution per message, message mapping and filtering, cache operations, database interactions, and handling message queues. Connectors and processors are central to its functionality; connectors manage data integration between Redpanda and various systems, while processors apply functions to messages for tasks such as transformation or filtering. With support for a wide range of technologies and formats, Redpanda Connect enhances data flow reliability and efficiency, making it a versatile tool for data engineering, stream processing, and enterprise integration.

Redpanda has announced the beta release of its Redpanda Cloud API, a REST API that empowers developers to manage Redpanda Cloud resources efficiently, either directly or through third-party tools. This API is available to all Redpanda Cloud customers using Dedicated and BYOC clusters, offering a modern streaming data platform that merges the performance of Apache Kafka® with cloud-native simplicity, providing scalability, fault tolerance, and low-latency processing. Aimed at DevOps-minded enterprises, the Redpanda Cloud API facilitates Infrastructure-as-Code approaches, enabling tasks like resource management, scaling, and automation. Early applications include CI/CD pipeline creation for resource management, self-service user onboarding, and BYOC cluster creation, with plans to expand access to Redpanda Serverless clusters. Users are encouraged to provide feedback during the beta phase to help shape future updates, with communication facilitated through the Redpanda Community on Slack.

In January 2023, Zafin appointed a new CTO to focus on product development and tackle the challenges of banking core modernization. Drawing from experience at IBM, the CTO explored Redpanda as a streamlined alternative to Apache Kafka for Zafin's data architecture needs. Zafin, a SaaS platform, aids banks in modernizing systems, enhancing customer experiences, and reducing costs by moving beyond outdated legacy systems. The company developed Zafin Integrate & Orchestrate (Zafin IO) to facilitate real-time data integration and orchestration between banks' core systems and its platform, with Redpanda at its core due to its simplicity, compatibility, and reliability. The platform's architecture includes Apache Nifi, Apache Flink, and Temporal, enabling seamless integration for large-scale financial institutions. Zafin's adoption of Redpanda has resulted in improved operations, reduced complexity, and significant event processing capability. The company is committed to expanding its use of Redpanda, driven by its successful integration experiences and the anticipated benefits of Redpanda Connect, which promises to further simplify data pipelines with its numerous connectors.