March 2026 Summaries

Redpanda Streaming 26.1 introduces a transformative approach to streaming by eliminating traditional trade-offs between speed, performance, and cost, with the launch of Redpanda One (R1) and the general availability of Cloud Topics. This update offers a flexible, multi-modal platform that allows for efficient scaling and management of streaming data, reducing operational complexity and costs by enabling direct writing to object storage like S3 or GCS without the need for additional clusters. The release also enhances security with Group-Based Access Control (GBAC), simplifying authorization across multiple users and clusters. In addition, updates to Leader Pinning and Remote Read Replicas improve scalability across regions while minimizing expenses, and other features such as Schema Registry contexts and JSON improvements for Iceberg Topics further bolster data management capabilities. The release represents a significant step forward in providing a single engine solution for diverse workloads, promising over 90% lower networking costs and maintaining reliability with local-disk broker features.

Cloud Topics, now in general availability with Redpanda Streaming 26.1, introduces an innovative architecture that uses object storage for data while storing metadata locally, optimizing for cost, latency, and performance. This system allows for the separation of data and metadata storage, bypassing traditional cross-AZ networking costs associated with Raft replication. Data is initially batched in memory before being uploaded to cloud storage as L0 files, which are later optimized into L1 files for efficient reading by a background process called the Reconciler. The use of placeholders in Raft logs ensures that transaction and idempotency guarantees are maintained. This new replication mechanism is designed to slash costs by leveraging object storage, and Redpanda provides resources for users to explore and provide feedback on Cloud Topics.

Deploying agentic AI systems requires high levels of confidence and trust, which many businesses find challenging due to the need for comprehensive governance, auditing, and observability. Redpanda's Agentic Data Plane (ADP) offers a solution by providing a framework to safely test, scale, and manage agentic systems, enhancing security and operational efficiency through its three-layer model comprising reasoning, action, and mediation layers. Data streaming is crucial for agentic systems due to shared needs such as resilience, distributed architecture, and real-time data interaction, addressing challenges like context maintenance, authorization, and governance. Companies are encouraged to start with well-understood problems when deploying agentic AI, allowing for easier validation and performance measurement, with potential applications in sectors like financial services, cybersecurity, and manufacturing. In manufacturing, for example, agents can assist in root cause analysis and highlight key equipment data for executives, providing proactive solutions to prevent equipment failures and improve operational insights.

Redpanda has developed and open-sourced the Redpanda AI SDK for Go to address the limitations of existing Go AI tooling, which often consists of thin wrappers, non-native ports of Python frameworks, or early-stage libraries. The SDK is designed to offer real provider portability, clean streaming, composable middleware, and a robust testing framework without relying on real API calls. It supports multiple AI service providers like OpenAI and AWS Bedrock and features a shared interface for model interactions, making it adaptable across different protocols with a layered interceptor system for easier observability and control. The SDK is integral to Redpanda's Agentic Data Plane, ensuring it remains maintained and production-ready, while also incorporating support for Model Context Protocol (MCP) and a simulated LLM framework for efficient testing.

Redpanda recently benchmarked the performance of NVIDIA Vera, a high-performance CPU based on the NVIDIA-designed Olympus core, and found it significantly outperformed other systems in handling Kafka-compatible workloads, delivering up to 5.5 times lower latency compared to AMD EPYC “Turin”. This advancement is particularly beneficial for industries leveraging agentic AI, as it allows data-intensive applications to be deployed closer to inference engines. In a series of tests, NVIDIA Vera demonstrated superior performance in streaming latencies and cross-core throughput, showing up to 73% faster performance in inter-core communication and completing multi-table joins faster than AMD alternatives. These results highlight NVIDIA Vera's capability to efficiently support CPU-intensive tasks like reinforcement learning and data processing at a data center scale, making it an ideal platform for Redpanda's mission-critical infrastructure software, which is designed to maximize CPU-based architecture efficiency.

Bring Your Own VPC (BYOVPC) for AWS is now generally available, offering users of Redpanda Cloud enhanced networking control by allowing them to manage their own virtual private cloud (VPC) while Redpanda deploys its data plane into it. BYOVPC is a step further from the Bring Your Own Cloud (BYOC) model, which manages the networking on behalf of users by creating and controlling resources like VPCs and subnets. With BYOVPC, users pre-provision these resources, granting them full ownership of their VPC, subnets, IAM roles, and security policies, while Redpanda operates within this defined environment without altering any configurations. The introduction of BYOVPC aligns with organizations needing granular control over their networking setup, especially those with stringent security and compliance requirements, and it integrates seamlessly with existing network topologies. Terraform is recommended for provisioning BYOVPC clusters, offering benefits like repeatability, auditability, and flexibility. This approach provides users with the ability to customize infrastructure while maintaining a minimal IAM footprint, and it supports features like PrivateLink and secrets management, ensuring secure and efficient cloud operations. Redpanda Cloud customers can access BYOVPC by contacting their account team or Redpanda Sales, and further documentation is available for setting up BYOVPC clusters on AWS.

Apache Iceberg has emerged as a preferred table format for teams looking to make streaming data queryable in a lakehouse, but the process often involves complex infrastructure and hidden costs. To simplify this, Redpanda has introduced the Iceberg output for Redpanda Connect, a component that allows for direct writing of streaming data to Iceberg tables from a declarative YAML pipeline. This integration enables data transformation, enrichment, and routing before the data reaches the lakehouse, supporting multiple data sources beyond Kafka streams. The Iceberg output leverages Redpanda Connect's ecosystem of inputs and processors, allowing for versatile data handling, including schema evolution and efficient resource usage. It supports integration with various REST catalog APIs and offers enterprise-grade governance, making it a lightweight yet powerful tool for managing data pipelines. The Iceberg output is designed for both high-throughput, standard Kafka-to-lakehouse processes and more complex pipelines involving non-Kafka sources, providing flexibility in data routing and transformation.

The podcast "Hello, Agent!" explores the practical implementation of AI agents in enterprise settings, emphasizing production-ready autonomous systems and addressing key challenges like data security, infrastructure, and scalability. Hosted by Redpanda founder and CEO Alex Gallego, the series features industry leaders who share their experiences and insights, such as Nicolas Dupont of Cyborg, who discusses securing vector databases for AI agents. The podcast aims to navigate the complex landscape of enterprise AI by combining technical expertise with strategic considerations, offering real-world examples and deep dives into current technologies like vector embeddings and cryptographic innovations. As AI becomes increasingly integrated into mission-critical systems, "Hello, Agent!" serves as a valuable resource for understanding how to deploy AI agents effectively and securely.