March 2023 Summaries

DeltaStream is a serverless stream processing platform designed to integrate seamlessly with various streaming storage services like Apache Kafka, AWS Kinesis, Confluent Cloud, AWS MSK, and Redpanda. It offers a SQL-based interface that allows users to create streaming applications, such as pipelines and materialized views, using a familiar language while also supporting custom computations through UDFs/UDAFs. The platform emphasizes ease of use by eliminating the need for managing clusters or scaling infrastructure, allowing independent scaling and recovery of queries. DeltaStream enhances data security and organization through features like fine-grained access control, namespacing, and a unified view of diverse streaming data sources. Additionally, it supports continuous queries and push notifications, enabling real-time interaction and secure data sharing across teams. With its robust capabilities, DeltaStream is positioned as a comprehensive solution for managing and processing streaming data efficiently.

Event streaming infrastructure is crucial for modern data operations, enabling organizations to gain real-time insights by processing streaming data. Streaming analytics systems, like Apache Flink and Kafka Stream, provide a compute layer for real-time data processing without their own storage, making them ideal for building real-time data pipelines. In contrast, streaming databases combine processing and storage, allowing for the creation of materialized views that serve real-time data but lack fine-grained control over queries, making them less suited for real-time pipeline construction. DeltaStream offers a unified platform that combines the advantages of both systems, allowing users to build, manage, and secure streaming applications with features like real-time materialized views, fine-grained query control, and familiar security protocols. DeltaStream's serverless service supports integration with existing streaming storage systems like Apache Kafka and AWS Kinesis, providing a comprehensive solution for organizing and securing streaming data while enabling real-time data sharing and application development.

DeltaStream is a unified serverless stream processing platform founded by Hojjat Jafarpour, designed to simplify and enhance the management, security, and processing of event streams using Apache Flink. Built to address the complexities and high costs of existing stream processing solutions, DeltaStream aims to offer an easy-to-use, automatically scaling, and comprehensive platform that works across multiple data streaming stores like Apache Kafka and Kinesis. The platform supports both stateless and stateful stream processing and includes features such as zero operations setup, multi-tenancy with query isolation, and robust security measures through access control and namespacing. DeltaStream is positioned to help businesses quickly gain insights from real-time data while reducing the engineering burden of managing infrastructure, with future expansions planned to further enhance its capabilities.