January 2021 Summaries

Redpanda, Inc. has announced the general availability of its Intelligent Data API Platform, along with securing $15.5 million in funding led by Lightspeed Venture Partners and GV, to further product development and market expansion. The platform is designed to address the complexities of operationalizing open-source components in real-time, data-streaming infrastructures, offering a unified approach to historical and real-time data management through a Kafka-API compatible system that promises 10x better performance and zero data loss without requiring code changes. Highlighting modern infrastructure needs, Redpanda leverages advancements in storage technology to provide a solution that simplifies enterprise data operations, as evidenced by its adoption by companies like Zenly, which praises its WebAssembly feature for creating a 'Data Firewall.' Founded in 2019 and based in San Francisco, Redpanda aims to support mission-critical applications by transforming data streams into reliable data products, thus improving enterprises' data handling and delivery to stakeholders.

At the Performance Summit on December 1st, a talk titled "Redpanda: A new storage engine built from the ground up for Kafka-API compatibility with Kernel bypass technologies for 10x lower tail latencies" was presented, detailing the creation of a new storage engine optimized for modern hardware. The presentation focused on the implementation of a thread per core architecture to ensure predictable tail latencies, and explored the challenges and performance benefits associated with adapting the existing Kafka-API protocol to this new engine. The talk emphasized significant performance improvements achieved through these optimizations, and viewers are encouraged to join the community discussion on Slack and Twitter for further engagement.