Benchmarking Diskless Topics: Part 1

Aiven's blog post presents an open-source benchmark for Diskless Kafka (KIP-1150), which significantly reduces cloud costs by leveraging direct-to-S3 topic storage, potentially cutting infrastructure expenses by over 94%. The benchmark, run on a cluster of six m8g.4xlarge machines across three availability zones, demonstrated a 1 GiB/s input and 3 GiB/s output workload, maintaining stable performance with less than 30% CPU usage and achieving ~1.6 seconds P99 end-to-end latency. This new approach offers substantial savings on disk and cross-zone replication costs while maintaining the capability to run classic and Diskless topics side-by-side in the same cluster. Although Diskless Kafka introduces a slight increase in latency due to reliance on object storage, the trade-off is offset by substantial cost savings, making it an attractive option for future Kafka deployments. The post emphasizes the need for realistic and open-source benchmarks, as they provide a reliable data point for evaluating Diskless topics' performance and encourage community involvement in the ongoing development of Apache Kafka's Diskless architecture.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.