Company
Date Published
Author
Filip Yonov
Word count
3174
Language
English
Hacker News points
3

Summary

Diskless Kafka is a new feature that allows Apache Kafka to write directly to object storage, eliminating the need for local disk replicas and reducing costs by up to 80%. This feature, part of KIP-1150, enables users to choose between low-latency streams and ultra-cheap backfills for their topics. Diskless is designed to keep everything - including low-latency streams and cheap backfills - inside one logical log. The feature has already been tested in production by Aiven, a managed service provider, where it showed significant cost savings and reliability improvements. Diskless is being released as an open-source fork called Inkless, which allows users to test the feature on their own clusters before it ships upstream. With Diskless, Kafka's original vision of being a friction-free data mobility layer becomes a reality, with organizations able to keep adding topics without having to build custom solutions or incur additional costs. The feature is expected to accelerate Kafka's adoption and improve its network effect, as users can now dial storage economics per topic, reducing the need for tiered storage solutions.