Introducing streaming ingest in Starburst Galaxy — Now in Private Preview

Starburst Galaxy introduces a new streaming ingest feature, currently in private preview, that allows for the continuous ingestion of data from Kafka-compliant topics into Apache Iceberg tables in near real-time while ensuring exactly-once processing guarantees. This solution simplifies the traditionally complex and costly process of building custom data ingestion systems, which often lack efficiency and scalability. Starburst's streaming ingest service offers a straightforward three-step setup, connecting Kafka sources to Iceberg tables and providing a transformation process for data mapping. This managed service is designed to handle large volumes of event data efficiently, making it a scalable and cost-effective alternative to existing solutions. Additionally, Starburst plans to enhance the service further by integrating it with materialized views systems to support the creation of declarative pipelines for data preparation.